dave
/
relstorage
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Move migration instructions online.

This commit is contained in:
Jason Madden 2016-12-20 09:58:05 -06:00
parent ae48f02ca3
commit 174409775b
No known key found for this signature in database
GPG Key ID: 349F84431A08B99E
16 changed files with 197 additions and 159 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -22,4 +22,3 @@ parts/
doc/_build
doc/__pycache__
doc/relstorage.*.rst
doc/changelog.rst

1
doc/changelog.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../CHANGES.rst

8
doc/conf.py
View File

@ -44,14 +44,6 @@ except ImportError:
os.system('%s generate_rst.py generate' % sys.executable)
if not os.path.exists('changelog.rst') and os.path.exists('../CHANGES.rst'):
print('Linking ../CHANGES.rst to changelog.rst')
if hasattr(os, 'symlink'):
os.symlink('../CHANGES.rst', 'changelog.rst')
else:
import shutil
shutil.copyfile('../CHANGES.rst', 'changelog.rst')
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.doctest',

2
doc/contents.rst
View File

@ -19,7 +19,7 @@ Contents:
migration
developing
changelog
HISTORY
old_history

7
doc/index.rst
View File

@ -65,3 +65,10 @@ Project URLs
.. toctree::
contents
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

21
doc/migration.rst
View File

@ -5,17 +5,12 @@
Sometimes RelStorage needs a schema modification along with a software
upgrade. Hopefully, this will not often be necessary.
Migration to RelStorage version 1.5 requires a schema upgrade.
See `migrate-to-1.5.txt`_.
.. toctree::
:maxdepth: 1
.. _`migrate-to-1.5.txt`: https://github.com/zodb/relstorage/blob/master/notes/migrate-to-1.5.txt
Migration to RelStorage version 1.4.2 requires a schema upgrade if
you are using a history-free database (meaning keep-history is false).
See `migrate-to-1.4.txt`_.
.. _`migrate-to-1.4.txt`: https://github.com/zodb/relstorage/blob/master/notes/migrate-to-1.4.txt
See the `notes subdirectory`_ if you are upgrading from an older version.
.. _`notes subdirectory`: https://github.com/zodb/relstorage/tree/master/notes
migration/migrate-to-1.5
migration/migrate-to-1.4
migration/migrate-to-1.1.2
migration/migrate-to-1.1.1
migration/migrate-to-1.1
migration/migrate-to-1.0

104
doc/migration/migrate-to-1.0.rst Normal file
View File

@ -0,0 +1,104 @@
====================
1.0 Beta Migration
====================
.. highlight:: sql
Use one of the following scripts to migrate from RelStorage 1.0 beta to
RelStorage 1.0. Alter the scripts to match the Python default encoding.
For example, if 'import sys; print sys.getdefaultencoding()' says the
encoding is "iso-8859-1", change all occurrences of 'UTF-8' or 'UTF8'
to 'ISO-8859-1'.
PostgreSQL 8.3 (using the psql command)::
ALTER TABLE transaction
ALTER username TYPE BYTEA USING (convert_to(username, 'UTF-8')),
ALTER description TYPE BYTEA USING (convert_to(description, 'UTF-8'));
PostgreSQL 8.2 and below (using the psql command)::
ALTER TABLE transaction
ALTER username TYPE BYTEA USING
(decode(replace(convert(username, 'UTF-8'), '\\', '\\\\'), 'escape')),
ALTER description TYPE BYTEA USING
(decode(replace(convert(description, 'UTF-8'), '\\', '\\\\'), 'escape'));
MySQL (using the mysql command)::
ALTER TABLE transaction
MODIFY username BLOB NOT NULL,
MODIFY description BLOB NOT NULL;
Oracle (using the sqlplus command)::
ALTER TABLE transaction ADD (
new_username RAW(500),
new_description RAW(2000),
new_extension RAW(2000));
UPDATE transaction
SET new_username = UTL_I18N.STRING_TO_RAW(username, 'UTF8'),
new_description = UTL_I18N.STRING_TO_RAW(description, 'UTF8'),
new_extension = extension;
ALTER TABLE transaction DROP (username, description, extension);
ALTER TABLE transaction RENAME COLUMN new_username TO username;
ALTER TABLE transaction RENAME COLUMN new_description TO description;
ALTER TABLE transaction RENAME COLUMN new_extension TO extension;
Migration From PGStorage to RelStorage
======================================
PostgreSQL::
-- Migration from PGStorage to RelStorage
-- Do all the work in a transaction
BEGIN;
-- Remove the commit_order information (RelStorage has a better solution).
DROP SEQUENCE commit_seq;
ALTER TABLE transaction DROP commit_order;
-- Make the special transaction 0 match RelStorage
UPDATE transaction SET username='system',
description='special transaction for object creation'
WHERE tid = 0;
-- Add the MD5 column and some more constraints.
ALTER TABLE object_state
ADD CONSTRAINT object_state_tid_check CHECK (tid > 0),
ADD CONSTRAINT object_state_prev_tid_fkey FOREIGN KEY (prev_tid)
REFERENCES transaction,
ADD COLUMN md5 CHAR(32);
UPDATE object_state SET md5=md5(state) WHERE state IS NOT NULL;
-- Replace the temporary tables used for packing.
DROP TABLE pack_operation;
DROP TABLE pack_transaction;
DROP TABLE pack_keep;
DROP TABLE pack_garbage;
CREATE TABLE pack_lock ();
CREATE TABLE object_ref (
zoid BIGINT NOT NULL,
tid BIGINT NOT NULL,
to_zoid BIGINT NOT NULL
);
CREATE INDEX object_ref_from ON object_ref (zoid);
CREATE INDEX object_ref_tid ON object_ref (tid);
CREATE INDEX object_ref_to ON object_ref (to_zoid);
CREATE TABLE object_refs_added (
tid BIGINT NOT NULL PRIMARY KEY
);
CREATE TABLE pack_object (
zoid BIGINT NOT NULL PRIMARY KEY,
keep BOOLEAN NOT NULL,
keep_tid BIGINT
);
CREATE INDEX pack_object_keep_zoid ON pack_object (keep, zoid);
-- Now commit everything
COMMIT;

16
notes/migrate-to-1.1.1.txt → doc/migration/migrate-to-1.1.1.rst
View File

@ -1,10 +1,16 @@
.. _migrate-to-1.1.1:
========================================================
Migrating from RelStorage version 1.1 to version 1.1.1
========================================================
.. highlight:: sql
Migrating from RelStorage version 1.1 to version 1.1.1
Before following these directions, first upgrade to the schema of
RelStorage version 1.1 by following the directions in "migrate-to-1.1.txt".
RelStorage version 1.1 by following the directions in :ref:`migrate-to-1.1`.
PostgreSQL:
PostgreSQL::
DROP TABLE pack_object;
CREATE TABLE pack_object (
@ -19,7 +25,7 @@ PostgreSQL:
WHERE keep = true;
MySQL:
MySQL::
DROP TABLE pack_object;
CREATE TABLE pack_object (
@ -31,7 +37,7 @@ MySQL:
CREATE INDEX pack_object_keep_zoid ON pack_object (keep, zoid);
Oracle:
Oracle::
DROP TABLE pack_object;
CREATE TABLE pack_object (

19
doc/migration/migrate-to-1.1.2.rst Normal file
View File

@ -0,0 +1,19 @@
.. _migrate-to-1.1.2:
==========================================================
Migrating from RelStorage version 1.1.1 to version 1.1.2
==========================================================
.. highlight:: sql
Before following these directions, first upgrade to the schema of
RelStorage version 1.1.1 by following the directions in :ref:`migrate-to-1.1.1`.
Only Oracle needs a schema update for this release::
DROP TABLE temp_pack_visit;
CREATE GLOBAL TEMPORARY TABLE temp_pack_visit (
zoid NUMBER(20) NOT NULL PRIMARY KEY,
keep_tid NUMBER(20)
);

17
notes/migrate-to-1.1.txt → doc/migration/migrate-to-1.1.rst
View File

@ -1,7 +1,12 @@
.. _migrate-to-1.1:
Migrating from RelStorage version 1.0 or 1.0.1 to version 1.1
===============================================================
Migrating from RelStorage version 1.0 or 1.0.1 to version 1.1
===============================================================
PostgreSQL:
.. highlight:: sql
PostgreSQL::
CREATE INDEX object_state_prev_tid ON object_state (prev_tid);
@ -31,17 +36,17 @@ PostgreSQL:
);
Users of PostgreSQL 8.2 and above should also drop the pack_lock table since
it has been replaced with an advisory lock:
it has been replaced with an advisory lock::
DROP TABLE pack_lock;
Users of PostgreSQL 8.1 and below still need the pack_lock table. If you
have deleted it, please create it again with the following statement:
have deleted it, please create it again with the following statement::
CREATE TABLE pack_lock ();
MySQL:
MySQL::
CREATE INDEX object_state_prev_tid ON object_state (prev_tid);
@ -65,7 +70,7 @@ MySQL:
) ENGINE = MyISAM;
Oracle:
Oracle::
CREATE INDEX object_state_prev_tid ON object_state (prev_tid);

22
notes/migrate-to-1.4.txt → doc/migration/migrate-to-1.4.rst
View File

@ -1,6 +1,11 @@
.. _migrate-to-1.4:
Migrating to RelStorage version 1.4.2
-------------------------------------
=======================================
Migrating to RelStorage version 1.4.2
=======================================
.. highlight:: sql
If you are using a history-free storage, you need to drop and re-create
the object_refs_added table. It contains only temporary state used during
@ -9,7 +14,7 @@ packing.
Do not make these changes to history-preserving databases.
PostgreSQL:
PostgreSQL::
DROP TABLE object_refs_added;
CREATE TABLE object_refs_added (
@ -17,7 +22,7 @@ PostgreSQL:
tid BIGINT NOT NULL
);
MySQL:
MySQL::
DROP TABLE object_refs_added;
CREATE TABLE object_refs_added (
@ -25,7 +30,7 @@ MySQL:
tid BIGINT NOT NULL
) ENGINE = MyISAM;
Oracle:
Oracle::
DROP TABLE object_refs_added;
CREATE TABLE object_refs_added (
@ -34,11 +39,12 @@ Oracle:
);
Migrating to RelStorage version 1.4
-----------------------------------
=====================================
Migrating to RelStorage version 1.4
=====================================
Before following these directions, first upgrade to the schema of
RelStorage version 1.1.2 by following the directions in "migrate-to-1.1.2.txt".
RelStorage version 1.1.2 by following the directions in :ref:`migrate-to-1.1.2`.
Only Oracle needs a change for this release. The Oracle adapter
now requires the EXECUTE permission on the DBMS_LOCK package.

26
notes/migrate-to-1.5.txt → doc/migration/migrate-to-1.5.rst
View File

@ -1,6 +1,8 @@
.. _migrate-to-1.5:
Migrating to RelStorage version 1.5
===================================
=====================================
Migrating to RelStorage version 1.5
=====================================
All databases need a schema migration for this release. This release
adds a state_size column to the object_state table, making it possible
@ -9,11 +11,13 @@ is intended for gathering statistics.
Please note that if you are using the history-free schema, you need to
first migrate to RelStorage 1.4.2 by following the instructions in
migrate-to-1.4.txt.
:ref:`migrate-to-1.4`.
.. highlight:: sql
PostgreSQL
----------
==========
1. Migrate the object_state table::
@ -50,7 +54,7 @@ PostgreSQL
END;
$blob_write$ LANGUAGE plpgsql;
BEGIN;
ALTER TABLE blob_chunk RENAME COLUMN chunk TO oldbytea;
ALTER TABLE blob_chunk RENAME COLUMN chunk TO oldbytea;
ALTER TABLE blob_chunk ADD COLUMN chunk OID;
UPDATE blob_chunk SET chunk = blob_write(oldbytea);
ALTER TABLE blob_chunk
@ -77,14 +81,18 @@ and then copying it back::
MySQL history-preserving
------------------------
========================
Execute::
ALTER TABLE object_state ADD COLUMN state_size BIGINT AFTER md5;
UPDATE object_state SET state_size = COALESCE(LENGTH(state), 0);
ALTER TABLE object_state MODIFY state_size BIGINT NOT NULL AFTER md5;
MySQL history-free
------------------
==================
Execute::
ALTER TABLE object_state ADD COLUMN state_size BIGINT AFTER tid;
UPDATE object_state SET state_size = COALESCE(LENGTH(state), 0);
@ -92,7 +100,9 @@ MySQL history-free
Oracle
------
======
Execute::
ALTER TABLE object_state ADD state_size NUMBER(20);
UPDATE object_state SET state_size = COALESCE(LENGTH(state), 0);

5
doc/HISTORY.rst → doc/old_history.rst
View File

@ -212,9 +212,10 @@
- Valery Suhomlinov discovered a problem with non-ASCII data in transaction
metadata. The problem has been fixed for all supported databases.
=================
PGStorage history
-----------------
=================
0.4
===

46
notes/migrate-to-1.0.txt
View File

@ -1,46 +0,0 @@
1.0 Beta Migration
Use one of the following scripts to migrate from RelStorage 1.0 beta to
RelStorage 1.0. Alter the scripts to match the Python default encoding.
For example, if 'import sys; print sys.getdefaultencoding()' says the
encoding is "iso-8859-1", change all occurrences of 'UTF-8' or 'UTF8'
to 'ISO-8859-1'.
PostgreSQL 8.3 (using the psql command):
ALTER TABLE transaction
ALTER username TYPE BYTEA USING (convert_to(username, 'UTF-8')),
ALTER description TYPE BYTEA USING (convert_to(description, 'UTF-8'));
PostgreSQL 8.2 and below (using the psql command):
ALTER TABLE transaction
ALTER username TYPE BYTEA USING
(decode(replace(convert(username, 'UTF-8'), '\\', '\\\\'), 'escape')),
ALTER description TYPE BYTEA USING
(decode(replace(convert(description, 'UTF-8'), '\\', '\\\\'), 'escape'));
MySQL (using the mysql command):
ALTER TABLE transaction
MODIFY username BLOB NOT NULL,
MODIFY description BLOB NOT NULL;
Oracle (using the sqlplus command):
ALTER TABLE transaction ADD (
new_username RAW(500),
new_description RAW(2000),
new_extension RAW(2000));
UPDATE transaction
SET new_username = UTL_I18N.STRING_TO_RAW(username, 'UTF8'),
new_description = UTL_I18N.STRING_TO_RAW(description, 'UTF8'),
new_extension = extension;
ALTER TABLE transaction DROP (username, description, extension);
ALTER TABLE transaction RENAME COLUMN new_username TO username;
ALTER TABLE transaction RENAME COLUMN new_description TO description;
ALTER TABLE transaction RENAME COLUMN new_extension TO extension;

13
notes/migrate-to-1.1.2.txt
View File

@ -1,13 +0,0 @@
Migrating from RelStorage version 1.1.1 to version 1.1.2
Before following these directions, first upgrade to the schema of
RelStorage version 1.1.1 by following the directions in "migrate-to-1.1.1.txt".
Only Oracle needs a schema update for this release:
DROP TABLE temp_pack_visit;
CREATE GLOBAL TEMPORARY TABLE temp_pack_visit (
zoid NUMBER(20) NOT NULL PRIMARY KEY,
keep_tid NUMBER(20)
);

48
notes/migrate.sql
View File

@ -1,48 +0,0 @@
-- Migration from PGStorage to RelStorage
-- Do all the work in a transaction
BEGIN;
-- Remove the commit_order information (RelStorage has a better solution).
DROP SEQUENCE commit_seq;
ALTER TABLE transaction DROP commit_order;
-- Make the special transaction 0 match RelStorage
UPDATE transaction SET username='system',
description='special transaction for object creation'
WHERE tid = 0;
-- Add the MD5 column and some more constraints.
ALTER TABLE object_state
ADD CONSTRAINT object_state_tid_check CHECK (tid > 0),
ADD CONSTRAINT object_state_prev_tid_fkey FOREIGN KEY (prev_tid)
REFERENCES transaction,
ADD COLUMN md5 CHAR(32);
UPDATE object_state SET md5=md5(state) WHERE state IS NOT NULL;
-- Replace the temporary tables used for packing.
DROP TABLE pack_operation;
DROP TABLE pack_transaction;
DROP TABLE pack_keep;
DROP TABLE pack_garbage;
CREATE TABLE pack_lock ();
CREATE TABLE object_ref (
zoid BIGINT NOT NULL,
tid BIGINT NOT NULL,
to_zoid BIGINT NOT NULL
);
CREATE INDEX object_ref_from ON object_ref (zoid);
CREATE INDEX object_ref_tid ON object_ref (tid);
CREATE INDEX object_ref_to ON object_ref (to_zoid);
CREATE TABLE object_refs_added (
tid BIGINT NOT NULL PRIMARY KEY
);
CREATE TABLE pack_object (
zoid BIGINT NOT NULL PRIMARY KEY,
keep BOOLEAN NOT NULL,
keep_tid BIGINT
);
CREATE INDEX pack_object_keep_zoid ON pack_object (keep, zoid);
-- Now commit everything
COMMIT;