relstorage/doc/old_history.rst

====================
 Historical Changes
====================

1.1.3 (2009-02-04)
==================

- In rare circumstances, ZODB can legitimately commit an object twice in a
  single transaction.  Fixed RelStorage to accept that.

- Auto reconnect to Oracle sometimes did not work because cx_Oracle was
  raising a different kind of exception than expected.  Fixed.

- Included LICENSE.txt in the source distribution.


1.1.2 (2009-01-27)
==================

- When both cache-servers and poll-interval are set, we now poll the
  cache for changes on every request.  This makes it possible to use
  a high poll-interval to reduce the database polling burden, yet
  every client can see changes immediately.

- Added the pack-dry-run option, which causes pack operations to only
  populate the pack tables with the list of objects and states to pack,
  but not actually pack.

- Refined the pack algorithm.  It was not removing as many object states
  as it should have.  As a bonus, there is now a method of adapters called
  fill_object_refs(), which could be useful for debugging.  It ensures the
  object_ref table is fully populated.

- Began using zc.buildout for development.

- Increased automated test coverage.

- Fixed KeyError reporting to not trip over a related KeyError while logging.


1.1.1 (2008-12-27)
==================

- Worked around MySQL performance bugs in packing.  Used temporary
  tables and another column in the pack_object table.  The
  other databases may benefit from the optimization as well.

- Applied an optimization using setinputsizes() to the Oracle code,
  bringing write speed back up to where it was in version 1.0.


1.1 (2008-12-19)
================

- Normalized poll-invalidation patches as Solaris' patch command would not
  accept the current format. The patches now apply with:
  patch -d lib/python/ZODB -p0 < poll-invalidation-1-zodb-3-X-X.patch

- In MySQL, Use DROP TABLE IF EXISTS instead of TRUNCATE to clear 'temp_store'
  because:

  - TRUNCATE has one page of caveats in the MySQL documentation.
  - TEMPORARY TABLEs have half a page of caveats when it comes to
    replication.
  - The end result is that 'temp_store' may not exist on the
    replication slave at the exact same time(s) it exists on the
    master.

- Implemented the database size query in MySQL, based on a patch from
  Kazuhiko Shiozaki.  Thanks!

- Optimized Oracle object retrieval by causing BLOBs to be sent inline
  when possible, based on a patch by Helge Tesdal.  By default, the
  optimization is activated automatically when cx_Oracle 5 is used.

- Updated the storage iterator code to be compatible with ZODB 3.9.
  The RelStorage tests now pass with the shane-poll-invalidations branch
  of ZODB 3.9.

- Added a translation of README.txt to Brazilian Portuguese by
  Rogerio Ferreira.  Thanks!


1.1c1
=====

- Added optional memcache integration.  This is useful when the connection
  to the relational database has high latency.

- Made it possible to set the pack and memcache options in zope.conf.

- Log more info when a KeyError occurs within RelStorage.


1.1b2
=====

- Made the MySQL locks database-specific rather than server-wide.  This is
  important for multi-database configurations.

- In the PostgreSQL adapter, made the pack lock fall back to table locking
  rather than advisory locks for PostgreSQL 8.1.

- Changed a query for following object references (used during packing)
  to work around a MySQL performance bug.  Thanks to Anton Stonor for
  discovering this.


1.1b1
=====

- Fixed the use of setup.py without setuptools.  Thanks to Chris Withers.

- Fixed type coercion of the transaction extension field.  This fixes
  an issue with converting databases.  Thanks to Kevin Smith for
  discovering this.

- Added logging to the pack code to help diagnose performance issues.

- Additions to the object_ref table are now periodically committed
  during pre_pack so that the work is not lost if pre_pack fails.

- Modified the pack code to pack one transaction at a time and
  release the commit lock frequently.  This should help large pack
  operations.

- Fixed buildout-based installation of the zodbconvert script.  Thanks to
  Jim Fulton.


1.0.1 (2008-03-11)
==================

- The speedtest script failed if run on a test database that has no tables.
  Now the script creates the tables if needed.  Thanks to Flavio Coelho
  for discovering this.

- Reworked the auto-reconnect logic so that applications never see
  temporary database disconnects if possible.  Thanks to Rigel Di Scala
  for pointing out this issue.

- Improved the log messages explaining database connection failures.

- Moved poll_invalidations to the common adapter base class, reducing the
  amount of code to maintain.


1.0 (2008-02-29)
================

- Added a utility for converting between storages called zodbconvert.


1.0c1
=====

- The previous fix for non-ASCII characters was incorrect.  Now transaction
  metadata is stored as raw bytes.  A schema migration is required; see
  notes/migrate-1.0-beta.txt.

- Integrated setuptools and made an egg.


1.0 beta
========

- Renamed to reflect expanding database support.

- Added support for Oracle 10g.

- Major overhaul with many scalability and reliability improvements,
  particularly in the area of packing.

- Moved to svn.zope.org and switched to ZPL 2.1.

- Made two-phase commit optional in both Oracle and PostgreSQL.  They
  both use commit_lock in such a way that the commit is not likely to
  fail in the second phase.

- Switched most database transaction isolation levels from serializable
  to read committed.  It turns out that commit_lock already provides
  the serializability guarantees we need, so it is safe to take advantage
  of the potential speed gains.  The one major exception is the load
  connection, which requires an unchanging view of the database.

- Stored objects are now buffered in a database table rather than a file.

- Stopped using the LISTEN and NOTIFY statements in PostgreSQL since
  they are not strictly transactional in the sense we require.

- Started using a prepared statement in PostgreSQL for getting the
  newest transaction ID quickly.

- Removed the code in the Oracle adapter for retrying connection attempts.
  (It is better to just reconfigure Oracle.)

- Added support for MySQL 5.0.

- Added the poll_interval option.  It reduces the frequency of database
  polls, but it also increases the potential for conflict errors on
  servers with high write volume.

- Implemented the storage iterator protocol, making it possible to copy
  transactions to and from FileStorage and other RelStorage instances.

- Fixed a bug that caused OIDs to be reused after importing transactions.
  Added a corresponding test.

- Made it possible to disable garbage collection during packing.
  Exposed the option in zope.conf.

- Valery Suhomlinov discovered a problem with non-ASCII data in transaction
  metadata.  The problem has been fixed for all supported databases.

=================
PGStorage history
=================


0.4
===

- Began using the PostgreSQL LISTEN and NOTIFY statements as a shortcut
  for invalidation polling.

- Removed the commit_order code.  The commit_order idea was intended to
  allow concurrent commits, but that idea is a little too ambitious while
  other more important ideas are being tested.  Something like it may
  come later.

- Improved connection management: only one database connection is
  held continuously open per storage instance.

- Reconnect to the database automatically.

- Removed test mode.

- Switched from using a ZODB.Connection subclass to a ZODB patch.  The
  Connection class changes in subtle ways too often to subclass reliably;
  a patch is much safer.

- PostgreSQL 8.1 is now a dependency because PGStorage uses two phase commit.

- Fixed an undo bug.  Symptom: attempting to examine the undo log revealed
  broken pickles.  Cause: the extension field was not being wrapped in
  psycopg2.Binary upon insert.  Solution: used psycopg2.Binary.
  Unfortunately, this doesn't fix existing transactions people have
  committed.  If anyone has any data to keep, fixing the old transactions
  should be easy.

- Moved from a private CVS repository to Sourceforge.
  See http://pgstorage.sourceforge.net .  Also switched to the MIT license.

- David Pratt added a basic getSize() implementation so that the Zope
  management interface displays an estimate of the size of the database.

- Turned PGStorage into a top-level package.  Python generally makes
  top-level packages easier to install.


0.3
===

- Made compatible with Zope 3, although an undo bug apparently remains.


0.2
===

- Fixed concurrent commits, which were generating deadlocks.  Fixed by
  adding a special table, "commit_lock", which is used for
  synchronizing increments of commit_seq (but only at final commit.)
  If you are upgrading from version 0.1, you need to change your
  database using the 'psql' prompt:

    create table commit_lock ();

- Added speed tests and an OpenDocument spreadsheet comparing
  FileStorage / ZEO with PGStorage.  PGStorage wins at reading objects
  and writing a lot of small transactions, while FileStorage / ZEO
  wins at writing big transactions.  Interestingly, they tie when
  writing a RAM disk.