 
## Migrating data from RethinkDB 1.7 and above

RethinkDB 1.7+ ships with `rethinkdb dump` and `rethinkdb restore`,
which provide a smoother migration experience: see the 
[complete guide on rethinkdb.com](http://rethinkdb.com/docs/migration)
for information on how to use these commands.


---

## Migrating data from RethinkDB 1.6.x and earlier (deprecated)

Due to improvements in the data format, you will need to migrate your existing
data files before upgrading to new versions of RethinkDB.

You can use this handy data migration script to easily migrate your cluster
to a new version.

### Steps for migrating your cluster

1. Run `import_export.rb --export --host HOST --port PORT`, where HOST:PORT
connects to a machine in your cluster.  This will write everything in your
cluster to disk, so make sure you have enough space.  
_Note_: The port is the port for clients (by default 28015).

2. Upgrade RethinkDB on all your servers, then move or delete all your
`rethinkdb_data` and `data` directories (they are incompatible with the new 
version).

3. Run `import_export.rb --import --host HOST --port PORT`.  This will re-import
all the data on disk into your cluster.  You must run this in the same directory
you ran the `import_export.rb --export` command, or else specify the directory
to read from with `import_export.rb --import DIR`.  You will also have to
specify the directory if you exported from more than one RethinkDB
cluster. 

**NOTE: If any of your tables use a nonstandard primary key (anything
except `id`), you need to tell the script about the nonstandard keys while
importing.  (We stupidly forgot to build a way to fetch the primary key of a
table into the original RDB protocol; this will be fixed in future versions.)
You can specify this with the `--primary-keys` option, which takes a JSON
document of the form `{"db_name.table_name": "key_name, ...}`.**

### Required dependencies

The migration script requires Ruby to be installed, as well as a few gems:

```bash
gem install json ruby_protobuf
```

__Note for OS X users:__ Due to an incompatibility between `ruby_protobuf` and Ruby 1.8.7, 
__Ruby 1.9 is required__ for the migration script.

### Simple example

Alice is running a single RethinkDB server on localhost with the default ports.
She does the following:

```
# Make a directory for the migration files we'll generate.
~ $ mkdir rethinkdb_migration
~ $ cd rethinkdb_migration
# Fetch the migration script from Github.
~/rethinkdb_migration $ curl -L \
                          http://raw.github.com/rethinkdb/rethinkdb/next/scripts/migration/import_export.rb \
                          > import_export.rb
# Run the migration script.
~/rethinkdb_migration $ ruby import_export.rb --help
Usage: import_export.rb [options]
    -h, --host HOST                  host to connect to (default localhost)
    -p, --port PORT                  port to connect on (default 28015)
    -e, --export                     export data from RethinkDB into new directory
    -i, --import [DIR]               import data into RethinkDB from directory DIR
    -k, --primary-keys JSON_SPEC     when importing data from old RethinkDB versions, you
                                     must specify all non-`id` primary keys as follows:
                                     '{"db_name.table_name": "key_name"}'
~/rethinkdb_migration $ ruby import_export.rb --export
2013-03-14T20:49:59-07:00 Exporting to rethinkdb_export_2013-03-14T20:49:59-07:00_23642...
2013-03-14T20:49:59-07:00 Connecting to localhost:28015...
2013-03-14T20:49:59-07:00 Connected with query language Query_Language_1...
2013-03-14T20:49:59-07:00 Exporting database test3...
2013-03-14T20:49:59-07:00 Exporting database test...
2013-03-14T20:49:59-07:00 Exporting table test.test2...
2013-03-14T20:49:59-07:00 Exporting table test.foo...
2013-03-14T20:49:59-07:00 Exporting table test.test...
2013-03-14T20:49:59-07:00 Exporting database test2...
2013-03-14T20:49:59-07:00 Exporting table test2.abc...
2013-03-14T20:49:59-07:00 Done!
~/rethinkdb_migration $ <SHUT DOWN RETHINKDB>
~/rethinkdb_migration $ <MOVE/DELETE RETHINKDB_DATA AND DATA DIRECTORY IN INSTANCE DIRECTORY>
~/rethinkdb_migration $ <UPGRADE RETHINKDB>
~/rethinkdb_migration $ <RESTART RETHINKDB>
~/rethinkdb_migration $ ruby import_export.rb --import
2013-03-14T20:50:31-07:00 No directory specified, defaulting to `rethinkdb_export_2013-03-14T20:49:59-07:00_23642`.
2013-03-14T20:50:31-07:00 Importing from rethinkdb_export_2013-03-14T20:49:59-07:00_23642...
2013-03-14T20:50:31-07:00 Connecting to localhost:28015...
2013-03-14T20:50:31-07:00 Connected with query language Query_Language_2...
2013-03-14T20:50:31-07:00 Importing database test3...
2013-03-14T20:50:31-07:00 Importing database test...
2013-03-14T20:50:31-07:00 Importing table test.test2 with primary key id...
2013-03-14T20:50:31-07:00 Importing table test.foo with primary key id...
2013-03-14T20:50:31-07:00 Importing table test.test with primary key id...
2013-03-14T20:50:31-07:00 Importing database test2...
2013-03-14T20:50:31-07:00 Importing table test2.abc with primary key id...
2013-03-14T20:50:32-07:00 Done!
~/rethinkdb_migration $
```

### Complex example

Bob is running two distinct RethinkDB clusters.  He can connect to one of them
at `newton:60715` and the other at `magneto:60515`.  The cluster on `newton`
has two tables `test.test` and `test.foo` that use as primary key `pkey` (instead of `id`).

```
# Make a directory for the migration files we'll generate.
~ $ mkdir rethinkdb_migration
~ $ cd rethinkdb_migration
# Fetch the migration script from Github.
~/rethinkdb_migration $ curl -L \
                          http://raw.github.com/rethinkdb/rethinkdb/next/scripts/migration/import_export.rb \
                          > import_export.rb
# Run the migration script.
~/rethinkdb_migration $ ruby import_export.rb --help
Usage: import_export.rb [options]
    -h, --host HOST                  host to connect to (default localhost)
    -p, --port PORT                  port to connect on (default 28015)
    -e, --export                     export data from RethinkDB into new directory
    -i, --import [DIR]               import data into RethinkDB from directory DIR
    -k, --primary-keys JSON_SPEC     when importing data from old RethinkDB versions, you
                                     must specify all non-`id` primary keys as follows:
                                     '{"db_name.table_name": "key_name"}'
~/rethinkdb_migration $ ruby import_export.rb -h newton -p 60715 --export
2013-03-14T21:22:01-07:00 Exporting to rethinkdb_export_2013-03-14T21:22:01-07:00_5048...
2013-03-14T21:22:01-07:00 Connecting to newton:60715...
2013-03-14T21:22:01-07:00 Connected with query language Query_Language_1...
2013-03-14T21:22:01-07:00 Exporting database test2...
2013-03-14T21:22:01-07:00 Exporting table test2.abc...
2013-03-14T21:22:01-07:00 Exporting database test3...
2013-03-14T21:22:01-07:00 Exporting database test...
2013-03-14T21:22:01-07:00 Exporting table test.test2...
2013-03-14T21:22:01-07:00 Exporting table test.test...
2013-03-14T21:22:01-07:00 Exporting table test.foo...
2013-03-14T21:22:01-07:00 Done!
~/rethinkdb_migration $ ruby import_export.rb -h magneto -p 60515 --export
2013-03-14T21:22:47-07:00 Exporting to rethinkdb_export_2013-03-14T21:22:47-07:00_5052...
2013-03-14T21:22:47-07:00 Connecting to magneto:60515...
2013-03-14T21:22:47-07:00 Connected with query language Query_Language_1...
2013-03-14T21:22:47-07:00 Exporting database test2...
2013-03-14T21:22:47-07:00 Exporting table test2.abc...
2013-03-14T21:22:47-07:00 Exporting database test3...
2013-03-14T21:22:47-07:00 Exporting database test...
2013-03-14T21:22:47-07:00 Exporting table test.test2...
2013-03-14T21:22:47-07:00 Exporting table test.test...
2013-03-14T21:22:47-07:00 Exporting table test.foo...
2013-03-14T21:22:47-07:00 Done!
~/rethinkdb_migration $ ls
import_export.rb
rethinkdb_export_2013-03-14T21:22:01-07:00_5048
rethinkdb_export_2013-03-14T21:22:47-07:00_5052
~/rethinkdb_migration $ <SHUTDOWN BOTH CLUSTERS>
~/rethinkdb_migration $ <MOVE/DELETE ALL RETHINKDB_DATA AND DATA DIRECTORY IN INSTANCE DIRECTORIES>
~/rethinkdb_migration $ <UPGRADE BOTH CLUSTERS>
~/rethinkdb_migration $ <RESTART BOTH CLUSTERS>
~/rethinkdb_migration $ ruby import_export.rb -h newton -p 60715 --import rethinkdb_export_2013-03-14T21:22:01-07:00_5048 \
                          --primary-keys '{"test.test": "pkey", "test.foo": "pkey"}'
2013-03-14T21:25:14-07:00 Importing from rethinkdb_export_2013-03-14T21:22:01-07:00_5048...
2013-03-14T21:25:14-07:00 Connecting to newton:60715...
2013-03-14T21:25:14-07:00 Connected with query language Query_Language_2...
2013-03-14T21:25:14-07:00 Importing database test2...
2013-03-14T21:25:14-07:00 Importing table test2.abc with primary key id...
2013-03-14T21:25:14-07:00 Importing database test3...
2013-03-14T21:25:14-07:00 Importing database test...
2013-03-14T21:25:14-07:00 Importing table test.test2 with primary key id...
2013-03-14T21:25:15-07:00 Importing table test.test with primary key pkey...
2013-03-14T21:25:15-07:00 Importing table test.foo with primary key pkey...
2013-03-14T21:25:15-07:00 Done!
~/rethinkdb_migration $ ruby import_export.rb -h magneto -p 60515 --import rethinkdb_export_2013-03-14T21:22:47-07:00_5052
2013-03-14T21:25:48-07:00 Importing from rethinkdb_export_2013-03-14T21:22:47-07:00_5052...
2013-03-14T21:25:48-07:00 Connecting to magneto:60515...
2013-03-14T21:25:48-07:00 Connected with query language Query_Language_2...
2013-03-14T21:25:48-07:00 Importing database test2...
2013-03-14T21:25:48-07:00 Importing table test2.abc with primary key id...
2013-03-14T21:25:48-07:00 Importing database test3...
2013-03-14T21:25:48-07:00 Importing database test...
2013-03-14T21:25:48-07:00 Importing table test.test2 with primary key id...
2013-03-14T21:25:48-07:00 Importing table test.test with primary key id...
2013-03-14T21:25:48-07:00 Importing table test.foo with primary key id...
2013-03-14T21:25:49-07:00 Done!
```
