README

groonga-schema

Groonga-schema is a Ruby library and tool to processes Groonga's schema.

% gem install groonga-schema

Here are command lines provided by groonga-schema:

• groonga-schema-diff: It reports difference between 2 schema.

groonga-schema-diff reports difference between 2 schema:

% groonga-schema-diff FROM_SCHEMA TO_SCHEMA

The output of groonga-schema-diff is a Groonga command list. It means that you can apply difference by processing the output of groonga-schema-diff by Groonga. The relation of them are similar to diff and patch.

The following example shows about it.

Here are sample schema:

current.grn:

table_create Logs TABLE_NO_KEY
column_create Logs timestamp COLUMN_SCALAR ShortText

new.grn:

table_create Logs TABLE_NO_KEY
column_create Logs timestamp COLUMN_SCALAR Time

In the current.grn schema, Logs.timestamp column's value type is ShortText. In the new.grn schema, it's Time.

Here is the output of groonga-schema-diff:

% groonga-schema-diff current.grn new.grn
column_create --flags "COLUMN_SCALAR" --name "timestamp_new" --table "Logs" --type "Time"
column_copy --from_name "timestamp" --from_table "Logs" --to_name "timestamp_new" --to_table "Logs"
column_rename --name "timestamp" --new_name "timestamp_old" --table "Logs"
column_rename --name "timestamp_new" --new_name "timestamp" --table "Logs"

column_remove --name "timestamp_old" --table "Logs"

The output Groonga command list does the followings:

1. Create a new column Logs.timestamp_new. The value type of the new column is Time not ShortText.

2. Copy data to Logs.timestamp_new from Logs.timestamp.

3. Rename Logs.timestamp to Logs.timestamp_old.

4. Rename Logs.timestamp_new to Logs.timestamp.

5. Remove Logs.timestamp_old.

It means that the output Groonga command list supports data migration.

Here is a sample database to show data migration:

% groonga DB_PATH dump
table_create Logs TABLE_NO_KEY
column_create Logs timestamp COLUMN_SCALAR ShortText

load --table Logs
[
["_id","timestamp"],
[1,"2016-08-16 00:00:01"],
[2,"2016-08-16 00:00:02"],
[3,"2016-08-16 00:00:03"],
[4,"2016-08-16 00:00:04"],
[5,"2016-08-16 00:00:05"]
]

You can apply the change by the following command lines:

% groonga-schema-diff current.grn new.grn > diff.grn
% groonga --file diff.grn DB_PATH

Or:

% groonga-schema-diff current.grn new.grn | groonga DB_PATH

Here is the sample database after applying the changes:

% groonga DB_PATH dump
table_create Logs TABLE_NO_KEY
column_create Logs timestamp COLUMN_SCALAR Time

load --table Logs
[
["_id","timestamp"],
[1,1471273201.0],
[2,1471273202.0],
[3,1471273203.0],
[4,1471273204.0],
[5,1471273205.0]
]

Logs.timestamp column's value type is changed to Time from ShortText and data are also converted.

You can also use groonga-schema-diff to remote databases.

The following command line shows difference between databases served at http://192.168.0.1:10041 and http://192.168.0.2:10041/:

% groonga-schema-diff \
    'http://192.168.0.1:10041/d/dump?dump_records=no' \
    'http://192.168.0.2:10041/d/dump?dump_records=no'

You can apply the output Groonga command list by groonga-client command provided by groonga-client-cli gem:

% groonga-client --host 192.168.0.1 diff.grn

The following command synchronizes schema at http://192.168.0.1:10041/ with schema at http://192.168.0.1:10042/:

% groonga-schema-diff \
    'http://192.168.0.1:10041/d/dump?dump_records=no' \
    'http://192.168.0.2:10041/d/dump?dump_records=no' |
  groonga-client --host 192.168.0.1

After this command line, the following command line outputs nothing because there are no difference between schema at http://192.168.0.1:10041/ and http://192.168.0.1:10042/:

% groonga-schema-diff \
    'http://192.168.0.1:10041/d/dump?dump_records=no' \
    'http://192.168.0.2:10041/d/dump?dump_records=no'
%

NOTE: You should use database carefully while applying changes. Because some tables and columns may be removed while applying changes. If you touch removed tables and/or columns from another threads, Groonga may be crashed. It's better that you reduce the max number of threads to 1 while applying changes like the following:

% echo thread_limit 1 | groonga-client --host 192.168.0.1
% groonga-schema-diff \
    'http://192.168.0.1:10041/d/dump?dump_records=no' \
    'http://192.168.0.2:10041/d/dump?dump_records=no' |
  groonga-client --host 192.168.0.1
% echo thread_limit 8 | groonga-client --host 192.168.0.1

NOTE: You can't use the thread_limit technique with groonga-httpd because groonga-httpd is multi-process model not multi-threading model. You need to reduce the number of workers by changing worker_processes to 1 in groonga-httpd.conf and reload the configuration file. You also need to increase the number of workers after you apply the changes.

TODO...

• Ruby

• English: groonga-talk@lists.sourceforge.net
• Japanese: groonga-dev@lists.sourceforge.jp

• English: Gitter:groonga/en
• Japanese: Gitter:groonga/ja

• Kouhei Sutou <kou@clear-code.com>

LGPLv2.1 or later. See doc/text/lgpl-2.1.txt for details.

(Kouhei Sutou has a right to change the license including contributed patches.)