#117 Migrations
Originally created by: sf-overlords
Created by: mramm
Created date: 2010-03-22 18:02:47.533000
Assigned to:mramm
Forge Migrations
There is a continual need to affect the schema of the forge databases.
The forge platform itself will continue to evolve, as will every app
built on top of it. This document outlines a proposed method to
address this need.
What is a migration?
A migration is a change of database 'schema'. For mongodb:
- create, rename or delete collections
- ensure or remove collection indices
- add or remove documents
- transform document contents
Defining a migration
Creating a migration is simple: you subclass Migration and define 2
methods "up" and "down". For example:
from flyway import Migration class V2(Migration): def up(connection): posts = connection.posts posts.create_index([("date", DESCENDING), ("author", ASCENDING)]) def down(connection): posts = connection.posts posts.drop_indices(['date', 'author'])
Additionally, a migration can specify what it depends on. This allows
each app to ensure a previous known state before it affects the mongo
datastore. Example:
from flyway import Migration class V3(Migration): depends_on = [V2, pyforge.migrations.V7] def up(connection): ... def down(connection): ...
By convention, the depends_on, if unspecified, will include the
previously numbered migration. In the V2 example, it would depends on
V1 of the same module namespace.
Migration organization
Each app on the forge, (including the platform itself), will contain a
migrations module, possibly specified in its config:
pyforge/pyforge/migrations.py
ForgeWiki/forgewiki/migrations.py
ForgeSCM/forgescm/migrations.py
...
Architecture
The top-level migration code will look something like this:
migration_set = inspect_modules_for_available_migrations()
def migrate(target_versions, databases, migration_set):
for db in databases:
current_versions = current_versions(db)
for migrate in planner(current_versions, desired_versions, migration_set):
migrate(db)
The planner method will recurse through the dependencies to return the
correct ordered set of migration function to run.
Usage
Each app's config will point to its migration module. As a result,
we'll wire up a "migrate" command to paster:
$ cd ForgeSCM $ paste migrate migrating to version 12 (currently version 7) +++++ done. $ paste migrate 8 migrating back to version 8 (currently version 12) ---- done. $
Originally by: sf-overlords
Post by rcopeland:
Work for review in rc/117