Commit Graph

61 Commits

Author SHA1 Message Date
Jason Madden a821ec1147
Clean up the configuring-your-application document.
Move plone to the end instead of the beginning.

Document how to use a configuration string instead of a file.

Better links to ZConfig, repoze.zodbconn, and the ZODB.config functions.

[skip ci]
2018-04-10 10:39:11 -05:00
Jason Madden 7015f597e3
Document that PyPy does not work with mysqlconnector and remove it from the matrix. 2018-02-02 09:29:42 -06:00
Jason Madden c7319f7d10
Bump the tested version of MySQL Connector.
When we updated PyPy on Travis to 5.8.0, it started producing errors
"Use multi-True when executing multiple statements", even when we are
demonstrably only executing one statement. E.g.,
https://travis-ci.org/zodb/relstorage/jobs/335720512#L923

If this doesn't fix it, then I plan to take PyPy+MySQL Connector out
of the test matrix.
2018-02-02 08:56:08 -06:00
Jason Madden 687251a043
Add mention of needing CFFI to compile from source. Fixes #187.
[skip ci]
2017-06-02 13:57:36 -05:00
Jason Madden 3b65f778e6
Test on ZODB 4.4
Fixes #169.

We need this version for IMultiCommitStorage and the registerDB fix in
ConflictResolvingStorage. Since these versions run on Python 2.7.8,
there's no motivation for trying to go back farther.
2017-04-11 09:27:46 -05:00
Jason Madden 02e1369159
Document cx_Oracle version constraints. Fixes #172. [skip ci] 2017-03-10 13:18:57 -06:00
Jason Madden 3e4009f2e6
record command [skip ci] 2017-02-01 15:51:58 -06:00
Jason Madden 48df4f440c
Performance docs for 2.1a1 vs 2.0 [skip ci] 2017-02-01 15:37:54 -06:00
Jason Madden 21887ef5f0
Version notes. [skip ci] 2017-01-27 11:11:53 -06:00
Jason Madden 8e7b49d2a6
Clean up mysqlconnector driver name. Clean up installation docs and driver statement docs. 2017-01-26 11:58:05 -06:00
Jason Madden 28f9b1ac56
Initial support for connector/python 2017-01-24 17:18:29 -06:00
Jason Madden 82dc954578
postgres 9.5+: use ON CONFLICT UPDATE to make temp permanent
Instead of a combination DELETE/INSERT statement. This gives us a 5-10%
performance increase.

BEFORE

"Transaction",                postgresql
"Add 1000 Objects",                17167
"Update 1000 Objects",             17314
"Read 1000 Warm Objects",          12575
"Read 1000 Cold Objects",          16714
"Read 1000 Hot Objects",           58357
"Read 1000 Steamin' Objects",    2124724

AFTER

"Transaction",                postgresql
"Add 1000 Objects",                18042
"Update 1000 Objects",             19380
"Read 1000 Warm Objects",          13611
"Read 1000 Cold Objects",          17970
"Read 1000 Hot Objects",           62971
"Read 1000 Steamin' Objects",    2069391
2017-01-24 10:40:25 -06:00
Jason Madden f712396c25
Update link to Plone blog post to be historic. Incorporate info from https://github.com/zodb/relstorage/issues/153 [skip ci] 2017-01-12 14:33:29 -06:00
Jason Madden 436fbefec1
More docs on setting up databases [skip ci] 2017-01-12 11:07:12 -06:00
Jason Madden cf003f6054
Data about mods to mysql config for reference [skip ci] 2016-12-22 16:55:48 -06:00
Jason Madden f1e7d650c1
format tweaks [skip ci] 2016-12-22 06:41:57 -06:00
Jason Madden a38f721c1e
Convert pack_policy.ods to RST. Fixes #83. [skip ci] 2016-12-22 06:33:27 -06:00
Jason Madden 64336db33b
Performance docs, moved from notes. [skip ci] 2016-12-21 14:50:07 -06:00
Jason Madden a1730bbdf8
Installation docs [skip ci] 2016-12-20 11:14:54 -06:00
Jason Madden c081f272f4
Doc updates. Bring in caching doc. 2016-12-20 10:32:12 -06:00
Jason Madden 174409775b
Move migration instructions online. 2016-12-20 09:58:05 -06:00
Jason Madden 402c965b9f Test on Appveyor (#145)
* First basic version of appveyor.yml for testing. Still need to configure databases. Lets just see if we build.

* stdint.h is not available on MSVC before 2010, but we're not using it anyway. myssqlclient won't compile on windows

* Try a define for broken c99 support in MSVC.

* Broader macro.

* Use old form of 0 initializing struct.

* More fixes for broken c99 support of MSVC.

* Attempt to setup the database.

* need /c

* give up on running the DB tests.

* produce wheels

* produce wheels

* one more try for mysql tests.

* Run the db script as part of the test, it seems services aren't started during install phase.

* MySQL tests are running, try adding postgres.

* badges

* right bin directory for postgres

* Skip one blob test on windows/py3
2016-12-11 11:06:27 -06:00
Jason Madden 877008293b
doc: tweak size/compression text for cache-local-dir
[skip ci]
2016-12-05 09:13:37 -06:00
Jason Madden d87be4f13a
Add version added note for cache-local-dir-write-max-size [skip ci] 2016-10-01 05:53:06 -05:00
Jason Madden 90a88f4c6b
Double the default delta size on CPython since we're so much more memory efficient. This helps the benchmarks and simulations; for example, orm-busy/100 climbs to 0.736 hit ratio from 0.699 (and it ran slightly faster too). 2016-09-27 10:37:27 -05:00
Jason Madden 4585c85950
Record hits with a delta size of 30000---much better. 2016-09-23 13:15:03 -05:00
Jason Madden 8107f3f3a7
Implement and document cache-local-dir-write-max-size 2016-09-20 12:25:06 -05:00
Jason Madden 949e8e5c44
Need python_full_version, not python_version. 2016-09-08 11:24:32 -05:00
Jason Madden ad62a870b2
Major optimizations for persistent cache files.
Typical operations for large caches will be 3x faster or more. Python 2
and <3.4 need more memory to write, though.

The benchmark is much better and more realistic which is what drove
these changes.

Introduced an option to not compress the local cache files; they aren't
compressed by default anymore because that was a big part of the
slowness.
2016-08-23 17:46:04 -05:00
Jason Madden b52752c332 Doc tweaks for max_allowed_packet. 2016-07-25 15:09:19 -05:00
Jason Madden daaa6e04b5 Doc updates stemming from the umysql size limit; add a better exception message for it. 2016-07-25 12:38:54 -05:00
Jason Madden e43a68e9cc Refactor DB support (#111)
* Begining refactor to fix #77.

* Split schema into specific packages.

* Refactor batchers for dbs.

* Move locker implementations to db-specific places

* split oidallocator.

* split scriptrunner.

* split txncontrol.

* split stats.

* split packundo.

* Factor ObjectMover. Not quite finished, Oracle still needs some attention and there's one more massive method that needs attention.

* oracle query formatting

* consolidate oraclescriptrunner

* split oracle connmanager.

* cleaner __init__ for oracle.

* clean up postgresql adapter.

* split mysql.

* refactor the big ObjectMover.move_from_temp method.

* Minor opts.

* minor fixes for oracle. test suite is clean for oracle now.
2016-07-25 12:08:48 -05:00
Jason Madden 8ad649511a Adapt code from NextThought to support umysqldb (#110)
* Adapt code from NextThought to implement umysqldb support. Fixes #79.
* exclude should be a sub-key of matrix.
* Force the right driver for the umysqldb tests.
* formatting and debuging.
* Use bytes, not bytearry, to avoid a crash in umysql.
* Tweak driver selection.
* test fixes for umysqldb.
2016-07-17 11:23:05 -05:00
Jason Madden 135ce741ea hardcode example until ZEO is released. [skip ci] 2016-07-16 13:45:51 -05:00
Jason Madden b5a1d73ba3 Add cache tracing docs. [skip ci] 2016-07-15 07:59:46 -05:00
Jason Madden 7c21418766 Another future option for cache usage [skip ci] 2016-07-08 18:00:07 -05:00
Jason Madden b25bff0824
Make the local cache smart enough to handle changes in compression settings in pickled files. 2016-07-08 16:22:59 -05:00
Jason Madden daaacdbc15
Document new options. 2016-07-08 14:28:16 -05:00
Jason Madden 353383b1af
Drop support for postgresql 8.1. This lets us simplify quite a bit of code. 2016-07-03 12:37:44 -05:00
Jason Madden 2c3f0ba1c0
clarify docs for pg8000 [skip ci] 2016-07-01 18:05:17 -05:00
Jason Madden 06bfe2da4b
Bump PG server for pg8000. 2016-07-01 17:44:55 -05:00
Jason Madden 4ba2b252d5
Initial support for pg8000
The major difference from psycopg2/cffi is that it doesn't have native
lobject support. This is easily worked around, but this draft is not
complete because it doesn't support chunks. Some refactoring will be
required to elegantly handle this.

Two minor differences:

- It doesn't support multiple statements in a single `cursor.execute()`
  call. Apparently it turns all statements into prepared statements,
  because the error is from the server. Easily worked around and
  unlikely to be a perfomance difference.
- It handles transaction isolation levels like most other drivers, in
  SQL. This could be handled more elegantly too.

One thing: The server spits out lots of "WARNING: not in a transaction".
If I enable statement logging, I see every `commit` immediately followed
by a `rollback`, which generates the warning. I'm not sure if that's
just us, or something the driver is doing differently (e.g, maybe
psycopg2/cffi always follows `commit` with `begin`? and then when we try
to `rollback` we're already in a transaction?). Annoying but harmless.
Still, would like to figure it out though.

Testing this pure-python driver on Python 3 also revealed a lot of
connection leaks, many of which I've fixed, but there are probably more.
This should be a win for PyPy. See also zopefoundation/ZODB#78.

The testing matrix is expanded to test the pure-python drivers on
CPython too and to test pg8000. (Travis will be the first time I run
psycopg2 tests; hope I didn't break anything!)

Fixes #85.
2016-07-01 17:10:36 -05:00
Jason Madden f46c2b3dee
Allow choosing drivers by name. Fixes #86 2016-07-01 11:27:13 -05:00
Jason Madden 671199a17c
Drop mention of drivers from the front page, too complicated and messy to maintain there [skip ci] 2016-06-30 16:08:18 -05:00
Jason Madden daa7dafcfd
Switch to recommending and testing mysqlclient on python 2. Fixes #96 2016-06-30 16:02:06 -05:00
Jason Madden a2db918343
Try to workaround the thread+coverage crash exhibited on travis CI. 2016-06-29 17:10:01 -05:00
Jason Madden 1db3154f30
remove most obvious vestiges of poll_interval. all tests pass. 2016-06-29 14:57:26 -05:00
Jason Madden 591979da3a
Move to version.txt for sharing with doc/conf.py and use with zest.releaser. [skip ci] 2016-06-28 12:33:02 -05:00
Jason Madden a2276f1501
Update version; we're going with 2.0 [skip ci] 2016-06-28 12:00:32 -05:00
Jason Madden e1c8880737
Add notes on why check16MObject can fail on MySQL and how to fix it. [skip ci] 2016-06-28 07:48:33 -05:00