Welcome to jaraco.mongodb documentation!

Decode support for JSON strings using ordered dictionaries and parsing MongoDB-specific objects like dates.

For example, if you have a JSON object representing a sort, you want to retain the order of keys:

>>> ob = decode('{"key1": 1, "key2": 2}')
>>> list(ob.keys())
['key1', 'key2']

Or if you want to query by a date, PyMongo needs a Python datetime object, which has no JSON representation, so this codec converts $date keys to date objects.

>>> ob = decode('{"$gte": {"$date": "2019-01-01"}}')
>>> ob['$gte']
datetime.datetime(2019, 1, 1, 0, 0)

This function is useful in particular if you’re accepting JSON queries over an HTTP connection and you don’t have the luxury of Javascript expressions like you see in the Mongo shell or Compass.

jaraco.mongodb.codec.decode(s, *, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=<function compose.<locals>.compose_two.<locals>.<lambda>>, **kw)

Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object.

object_hook is an optional function that will be called with the result of any object literal decode (a dict). The return value of object_hook will be used instead of the dict. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting).

object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders. If object_hook is also defined, the object_pairs_hook takes priority.

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

parse_int, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).

parse_constant, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN. This can be used to raise an exception if invalid JSON numbers are encountered.

To use a custom JSONDecoder subclass, specify it with the cls kwarg; otherwise JSONDecoder is used.

The encoding argument is ignored and deprecated.

jaraco.mongodb.codec.maybe_date(obj)
>>> maybe_date({"$date": "2019-01-01"})
datetime.datetime(2019, 1, 1, 0, 0)

Backslash-escape text such that it is safe for MongoDB fields, honoring Restrictions on Field Names.

>>> decode(encode('my text with dots...'))
'my text with dots...'
>>> decode(encode(r'my text with both \. and literal .'))
'my text with both \\. and literal .'
>>> decode(encode('$leading dollar'))
'$leading dollar'
jaraco.mongodb.fields.decode(encoded)
jaraco.mongodb.fields.encode(text)
jaraco.mongodb.fields.unescape(match)

Helper functions to augment PyMongo

jaraco.mongodb.helper.connect(uri, factory=<class 'pymongo.mongo_client.MongoClient'>)

Use the factory to establish a connection to uri.

jaraco.mongodb.helper.connect_db(uri, default_db_name=None, factory=<class 'pymongo.mongo_client.MongoClient'>)

Use pymongo to parse a uri (possibly including database name) into a connected database object.

This serves as a convenience function for the common use case where one wishes to get the Database object and is less concerned about the intermediate MongoClient object that pymongo creates (though the connection is always available as db.client).

>>> db = connect_db(
...     'mongodb://mongodb.localhost/mydb?readPreference=secondary')
>>> db.name
'mydb'
>>> db.client.read_preference
Secondary(...)

If no database is indicated in the uri, fall back to default.

>>> db = connect_db('mongodb://mgo/', 'defaultdb')
>>> db.name
'defaultdb'

The default should only apply if no db was present in the URI.

>>> db = connect_db('mongodb://mgo/mydb', 'defaultdb')
>>> db.name
'mydb'
jaraco.mongodb.helper.connect_gridfs(uri, db=None)

Construct a GridFS instance for a MongoDB URI.

jaraco.mongodb.helper.filter_warnings()
jaraco.mongodb.helper.get_collection(uri)

Migration support as features in MongoWorld 2016 From the Polls to the Trolls.

The Manager class provides a general purpose support for migrating documents to a target version through a series of migration functions.

class jaraco.mongodb.migration.Manager(target_version)

Bases: object

A manager for facilitating the registration of migration functions and applying those migrations to documents.

To use, implement migration functions to and from each adjacent version of your schema and decorate each with the register classmethod. For example:

>>> @Manager.register
... def v1_to_2(manager, doc):
...     doc['foo'] = 'bar'
>>> @Manager.register
... def v2_to_3(manager, doc):
...     doc['foo'] = doc['foo'] + ' baz'

Note that in addition to the document, the migration manager is also passed to the migration function, allowing for other context to be made available during the migration.

To create a manager for migrating documents to version 3:

>>> mgr = Manager(3)

Then, use the manager to migrate a document to a target version.

>>> v1_doc = dict(version=1, data='stub')
>>> v3_doc = mgr.migrate_doc(v1_doc)
>>> v3_doc['version']
3
>>> v3_doc['foo']
'bar baz'

Note that the document is modified in place:

>>> v1_doc is v3_doc
True
>>> Manager._upgrade_funcs.clear()
migrate_doc(doc)

Migrate the doc from its current version to the target version and return it.

classmethod register(func)

Decorate a migration function with this method to make it available for migrating cases.

version_attribute_name = 'version'
class jaraco.mongodb.oplog.NiceRepr(orig)

Bases: object

Adapt a Python representation of a MongoDB object to make it appear nicely when rendered as a string.

>>> messy_doc = collections.OrderedDict([
...     ('ts', bson.Timestamp(1111111111, 30),
... )])
>>> print(NiceRepr(messy_doc))
{"ts": {"$timestamp": {"t": 1111111111, "i": 30}}}
class jaraco.mongodb.oplog.NullResumeFile

Bases: object

read()
save(ts)
class jaraco.mongodb.oplog.Oplog(coll)

Bases: object

find_params = {}
get_latest_ts()
has_ops_before(ts)

Determine if there are any ops before ts

query(spec)
since(ts)

Query the oplog for items since ts and then return

class jaraco.mongodb.oplog.RenameSpec(old_ns, new_ns)

Bases: object

affects(ns)
classmethod from_spec(string_spec)

Construct RenameSpec from a pair separated by equal sign (‘=’).

class jaraco.mongodb.oplog.Renamer

Bases: list

>>> specs = [
...      'a=b',
...      'alpha=gamma',
...  ]
>>> renames = Renamer.from_specs(specs)
>>> op = dict(ns='a.a')
>>> renames(op)
>>> op['ns']
'b.a'
>>> renames.affects('alpha.foo')
True
>>> renames.affects('b.gamma')
False
affects(ns)

Return True if this renamer affects the indicated namespace.

classmethod from_specs(specs)
invoke(op)

Replace namespaces in op based on RenameSpecs in self.

class jaraco.mongodb.oplog.ResumeFile

Bases: str

read()

Read timestamp from file.

save(ts)

Save timestamp to file.

class jaraco.mongodb.oplog.TailingOplog(coll)

Bases: jaraco.mongodb.oplog.Oplog

find_params = {'cursor_type': 34, 'oplog_replay': True}
since(ts)

Tail the oplog, starting from ts.

class jaraco.mongodb.oplog.Timestamp(time, inc)

Bases: bson.timestamp.Timestamp

dump(stream)

Serialize self to text stream.

Matches convention of mongooplog.

classmethod for_window(window)

Given a timedelta window, return a timestamp representing that time.

classmethod load(stream)

Load a serialized version of self from text stream.

Expects the format used by mongooplog.

classmethod wrap(orig)

Wrap an original timestamp as returned by a pymongo query with a version of this class.

jaraco.mongodb.oplog.applies_to_ns(op, ns)
jaraco.mongodb.oplog.apply(db, op)

Apply operation in db

jaraco.mongodb.oplog.delta_from_seconds(seconds)
jaraco.mongodb.oplog.main()
jaraco.mongodb.oplog.parse_args(*args, **kwargs)

Parse the args for the command.

It should be possible for one to specify ‘–ns’, ‘-x’, and ‘–rename’ multiple times:

>>> args = parse_args(['--ns', 'foo', 'bar', '--ns', 'baz'])
>>> args.ns
['foo', 'bar', 'baz']
>>> parse_args(['-x', '--exclude']).exclude
[]
>>> renames = parse_args(['--rename', 'a=b', '--rename', 'b=c']).rename
>>> len(renames)
2
>>> type(renames)
<class 'jaraco.mongodb.oplog.Renamer'>
jaraco.mongodb.oplog.string_none(value)

Convert the string ‘none’ to None

jaraco.mongodb.query.compat_explain(cur)

Simulate MongoDB 3.0 explain result on prior versions. http://docs.mongodb.org/v3.0/reference/explain-results/

jaraco.mongodb.query.project(*args, **kwargs)

Build a projection for MongoDB.

Due to https://jira.mongodb.org/browse/SERVER-3156, until MongoDB 2.6, the values must be integers and not boolean.

>>> project(a=True) == {'a': 1}
True

Once MongoDB 2.6 is released, replace use of this function with a simple dict.

jaraco.mongodb.query.upsert_and_fetch(coll, doc, **kwargs)

Fetch exactly one matching document or upsert the document if not found, returning the matching or upserted document.

See https://jira.mongodb.org/browse/SERVER-28434 describing the condition where MongoDB is uninterested in providing an upsert and fetch behavior.

>>> instance = getfixture('mongodb_instance').get_connection()
>>> coll = instance.test_upsert_and_fetch.items
>>> doc = {'foo': 'bar'}
>>> inserted = upsert_and_fetch(coll, doc)
>>> inserted
{...'foo': 'bar'...}
>>> upsert_and_fetch(coll, doc) == inserted
True

A MongoDB-backed CherryPy session store.

Although this module requires CherryPy, it does not impose the requirement on the package, as any user of this module will already require CherryPy.

To enable these sessions, your code must call Session.install() and then configure the CherryPy endpoint to use MongoDB sessions. For example:

jaraco.mongodb.sessions.Session.install()

session_config = {
        'sessions.on': True,
        'sessions.storage_type': 'MongoDB',
        'sessions.database': pymongo.MongoClient().database,
}
config = {
        '/': session_config,
}

cherrypy.quickstart(..., config=config)

The jaraco.modb module implements the codec interface, so may be used to encode more complex objects in the session:

session_config.update({
        'sessions.codec': jaraco.modb,
})
exception jaraco.mongodb.sessions.LockTimeout

Bases: RuntimeError

class jaraco.mongodb.sessions.NullCodec

Bases: object

decode(data)
encode(data)
class jaraco.mongodb.sessions.Session(id, **kwargs)

Bases: cherrypy.lib.sessions.Session

A MongoDB-backed CherryPy session store. Takes the following params:

database: the pymongo Database object. collection_name: The name of the collection to use in the db. codec: An object with ‘encode’ and ‘decode’ methods, used to encode

objects before saving them to MongoDB and decode them when loaded from MongoDB.

lock_timeout: A timedelta or numeric seconds indicating how long

to block acquiring a lock. If None (default), acquiring a lock will block indefinitely.

acquire_lock()

Acquire the lock. Blocks indefinitely until lock is available unless lock_timeout was supplied. If the lock_timeout elapses, raises LockTimeout.

codec = <jaraco.mongodb.sessions.NullCodec object>

by default, objects are passed directly to MongoDB

property collection
classmethod install()

Add this session to the cherrypy session handlers. CherryPy looks for session classes in vars(cherrypy.lib.sessions) with a name in title-case followed by “Session”.

release_lock()
setup_expiration()

Use pymongo TTL index to automatically expire sessions.

jaraco.mongodb.sharding.create_db_in_shard(db_name, shard, client=None)

In a sharded cluster, create a database in a particular shard.

jaraco.mongodb.sharding.get_ids(collection)
class jaraco.mongodb.timers.NeverExpires

Bases: object

expired()
class jaraco.mongodb.timers.Timer(expiration)

Bases: object

A simple timer that will indicate when an expiration time has passed.

classmethod after(elapsed)

Return a timer that will expire after elapsed passes.

expired()
jaraco.mongodb.uri.join(base, new)

Use urllib.parse to join the MongoDB URIs. Registers the MongoDB scheme first.

Indices and tables