Yet Another Document Mapper¶
It’s small and simple ODM for use with MongoDB.
Quick start¶
import pymongo
from yadm import Database, Document, fields
# Create model
class BlogPost(Document):
__collection__ = 'blog_posts'
title = fields.StringField()
body = fields.StringField()
# Create post
post = BlogPost()
post.title = 'Small post'
post.body = 'Bla-bla-bla...'
# Connect to database
client = pymongo.MongoClient("localhost", 27017)
db = Database(client, 'test')
# Insert post to database
db.insert(post)
# Query posts
qs = db.get_queryset(BlogPost).find({'title': {'$regex': '^s.*'}})
assert qs.count() > 0
for post in qs:
assert post.title.startswith('s')
# Query one post
post = db.get_queryset(BlogPost).find_one({'title': 'Small post'})
# Change post
post.title = 'Bla-bla-bla title'
# Save changed post
db.save(post)
CHANGES¶
1.2 (2016-XX-XX)¶
- Add
QuerySet.ids
method for get only documents id’s from queryset; - Add
Money.total_cents
method andMoney.from_cents
classmethod;
1.1 (2016-04-26)¶
- Add cacheing on queryset level and use it for
Add mongo aggregation framework support;
- Add
exc
argument to QuerySet.find_one
for raise specified exception if not found;
- Add
- Add
multi
argument to QuerySet.remove
;
- Add
Deprecate
QuerySet.find_one
Refactoring.
1.0 (2015-11-14)¶
- Change document structure. No more bad
BaseDocument.__data__
attribute: BaseDocument.__raw__
: raw data from mongo;BaseDocument.__cache__
: cached objects, casted with fields;BaseDocument.__changed__
: changed objects.
- Change document structure. No more bad
- Changes api for custom fields:
- Not more need create field descriptors for every field;
prepare_value
called only for setattr;to_mongo
called only for save objects to mongo;from_mongo
called only for load values fromBaseDocument.__raw__
;- Remove Field.default attribute. Use
Field.get_default
method; - Add
get_if_not_loaded
andget_if_attribute_not_set
method; - By default raise
NotLoadedError
if field not loaded from projection;
- Changes in
ReferenceField
: - Raise
BrokenReference
if link is bloken; - Raise
NotBindingToDatabase
if document not saved to database;
- Raise
- Changes in
smart_null keyword for
Field
;Fields in document must be instances (not classes!);
Remove ArrayContainer and ArrayContainerField;
Remove old MapIntKeysField and MapObjectIdKeysField. Use new
MapCustomKeysField
;Add
Database.update_one
method for run simple update query with specified document;Add
QuerySet.distinct
;serialize.from_mongo
now accept not_loaded sequence with filed names who must mark as not loaded, parent and name;serialize.to_mongo
do not callFieldDescriptor.__set__
;Fakers! Subsystem for generate test objects;
Tests now use pytest;
And more, and more...
API documentation¶
API¶
API documentation
Database¶
This module for provide work with MongoDB database.
import pymongo
from yadm.database import Database
from mydocs import Doc
client = pymongo.MongoClient("localhost", 27017)
db = Database(self.client, 'test')
doc = Doc()
db.insert(doc)
doc.arg = 13
db.save(doc)
qs = db.get_queryset(Doc).find({'arg': {'$gt': 10}})
for doc in qs:
print(doc)
-
class
yadm.database.
Database
(client, name)¶ Main object who provide work with database.
Parameters: - client (pymongo.Client) – database connection
- name (str) – database name
-
aggregate
(document_class, *, pipeline=None)¶ Return aggregator for use aggregation framework.
Parameters: - document_class –
yadm.documents.Document
- pipeline (list) – initial pipeline
- document_class –
-
bulk
(document_class, ordered=False, raise_on_errors=True)¶ Return Bulk.
Parameters: - document_class (MetaDocument) – class of documents fo bulk
- ordered (bool) – create ordered bulk (default False)
- raise_on_errors (bool) – raise BulkWriteError exception if write errors (default True)
Context manager:
- with db.bulk(Doc) as bulk:
- bulk.insert(doc_1) bulk.insert(doc_2)
-
get_queryset
(document_class, *, cache=None)¶ Return queryset for document class.
Parameters: - document_class –
yadm.documents.Document
- cache – cache for share with other querysets
This create instance of
yadm.queryset.QuerySet
with presetted document’s collection information.- document_class –
-
insert
(document)¶ Insert document to database.
Parameters: document (Document) – document instance for insert to database It’s bind new document to database set
_id
.
-
reload
(document, new_instance=False)¶ Reload document.
Parameters: - document (Document) – instance for reload
- new_instance (bool) – if True return new instance of document, else change data in given document (default: False)
-
remove
(document)¶ Remove document from database.
Parameters: document (Document) – instance for remove from database
Documents¶
Basic documents classes for build models.
class User(Document):
__collection__ = 'users'
first_name = fields.StringField()
last_name = fields.StringField()
age = fields.IntegerField()
All fields placed in yadm.fields
package.
-
class
yadm.documents.
MetaDocument
(cls, name, bases, cls_dict)¶ Metaclass for documents.
-
class
yadm.documents.
BaseDocument
(**kwargs)¶ Base class for all documents.
-
__raw__
¶ Dict with raw data from mongo
-
__cache__
¶ Dict with cached objects, casted with fields
-
__changed__
¶ Dict with changed objects
-
__data__
¶ Deprecated! For backward compatibility only!
Old way to storing data in documents. Now equal to
__raw__
.
-
__debug_print__
()¶ Print debug information.
-
__fake__
(values, faker, depth)¶ Fake data customizer.
-
-
class
yadm.documents.
Document
(**kwargs)¶ Class for build first level documents.
-
__collection__
¶ Name of MongoDB collection
-
_id
¶ Mongo object id (
bson.ObjectId
)
-
__db__
¶ Internal attribute contain instance of
yadm.database.Database
for realizeyadm.fields.references.ReferenceField
. It bind inyadm.database.Database
oryadm.queryset.QuerySet
.
-
__qs__
¶ Documents gets from this queryset
-
-
class
yadm.documents.
DocumentItemMixin
¶ Mixin for custom all fields values, such as
EmbeddedDocument
,yadm.fields.containers.Container
.-
__parent__
¶ Parent object.
assert doc.embedded_doc.__parent__ is doc assert doc.list[13].__parent__ is doc.list
-
__name__
¶ assert doc.list.__name__ == 'list' assert doc.list[13].__name__ == 13
-
__db__
¶ Database object.
assert doc.f.l[0].__db__ is doc.__db__
-
__document__
¶ Root document.
assert doc.f.l[0].__document__ is doc
-
__field_name__
¶ Dotted field name for MongoDB opperations, like as $set, $push and other...
assert doc.f.l[0].__field_name__ == 'f.l.0'
-
__get_value__
(document)¶ Get value from document with path to self.
-
__path__
¶ Path to root generator.
assert list(doc.f.l[0].__path__) == [doc.f.l[0], doc.f.l, doc.f]
-
__path_names__
¶ Path to root generator.
assert list(doc.f.l[0].__path__) == [0, 'l', 'f']
-
__qs__
¶ Queryset object.
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
class
yadm.documents.
EmbeddedDocument
(**kwargs)¶ Class for build embedded documents.
Serializers and deserializers¶
Functions for serialize and deserialize data.
-
yadm.serialize.
from_mongo
(document_class, data, not_loaded=(), parent=None, name=None)¶ Deserialize MongoDB data to document.
Parameters: - document_class – document class
- data (dict) – data from MongoDB
- not_loaded (list) – fields, who marked as not loaded
- parent – parent for new document
- name (str) – name for new document
-
yadm.serialize.
to_mongo
(document, exclude=(), include=None)¶ Serialize document to MongoDB data.
Parameters: - document (BaseDocument) – document for serializing
- exclude (list) – exclude fields
- include (list) – include only fields (all by default)
Queryset¶
-
class
yadm.queryset.
BaseQuerySet
(db, document_class, *, cache=None, criteria=None, projection=None, sort=None, slice=None, read_preference=None)¶ Query builder.
Parameters: - db –
- document_class –
- cache –
- criteria (dict) –
- projection (dict) –
- sort (list) –
- slice (slice) –
- read_preference (int) –
-
cache
¶ Queryset cache object.
-
copy
(*, cache=None, criteria=None, projection=None, sort=None, slice=None, read_preference=None)¶ Copy queryset with new parameters.
Only keywords arguments is alowed. Parameters simply replaced with given arguments.
Parameters: - cache –
- criteria (dict) –
- projection (dict) –
- sort (list) –
- slice (slice) –
- read_preference (int) –
Returns: new
yadm.queryset.QuerySet
object
-
fields
(*fields)¶ Get only setted fields.
Update projection with fields.
Parameters: fields (str) – Returns: new yadm.queryset.QuerySet
qs('field', 'field2')
-
fields_all
()¶ Clear projection.
-
find
(criteria=None, projection=None)¶ Return queryset copy with new criteria and projection.
Parameters: - criteria (dict) – update queryset’s criteria
- projection (dict) – update queryset’s projection
Returns: new
yadm.queryset.QuerySet
qs({'field': {'$gt': 3}}, {'field': True})
-
read_preference
(read_preference)¶ Setup readPreference.
Return new QuerySet instance.
-
sort
(*sort)¶ Return queryset with sorting.
Parameters: sort (tuples) – tuples with two items: (‘field_name’, sort_order_as_int). qs.sort(('field_1', 1), ('field_2', -1))
Bulk queries¶
-
class
yadm.bulk.
Bulk
(db, document_class, ordered=False, raise_on_errors=True)¶ Bulk object.
Parameters: - db (Database) – Database instance
- document_class (MetaDocument) – document class for collection
- ordered (bool) – create ordered bulk (default False)
- raise_on_errors (bool) – raise BulkWriteError exception if write errors (default True)
Context manager example:
- with db.bulk(Doc) as bulk:
- bulk.insert(doc_1) bulk.insert(doc_2)
-
execute
()¶ Execute the bulk query.
Returns: BulkResult
instance
-
insert
(document)¶ Add insert document to bulk.
Parameters: document (Document) – document for insert Warning
This unlike
Database.insert
! Currently, it is not bind objects to database and set id.
-
class
yadm.bulk.
BulkResult
(bulk, raw)¶ Object who provide result of Bulk.execute().
-
n_inserted
¶ Provide nInserted from raw result.
-
n_modified
¶ Provide nModified from raw result.
-
n_removed
¶ Provide nRemoved from raw result.
-
n_upserted
¶ Provide nUpserted from raw result.
-
write_errors
¶ Provide writeErrors from raw result.
-
Mongo Aggregation Framework¶
Mongo Aggregation Framework helper.
cur = db.aggregate(Doc).match({'i': {'$gt': 13}}).project(a='$i').limit(8)
Join¶
-
class
yadm.join.
Join
(qs)¶ Helper for build client-side joins.
# Doc.ref is instance of ReferenceField qs = db(Doc).find({'k': 1}) # queryset filter join = qs.join('ref') # create join query in this place for doc in join: print(doc.ref) # do not create query to database
-
get_queryset
(field_name)¶ Return queryset for joined objects.
-
join
(*field_names)¶ Do manual join.
-
Fields¶
This package contain all fields.
Base fields¶
Base classes for build database fields.
-
class
yadm.fields.base.
NotLoadedError
¶ Raise if value marked as not loaded.
doc = db(Doc).fields('a').find_one() try: doc.b except NotLoadedError: print("raised!")
-
class
yadm.fields.base.
FieldDescriptor
(name, field)¶ Base desctiptor for fields.
-
name
¶
Name of field
-
field
¶
Field instance for this desctiptor
-
__delete__
(instance)¶ Mark document’s key as not set.
-
__get__
(instance, owner)¶ Get python value from document.
Lookup in __changed__;
Lookup in __cache__;
Lookup in __raw__:
- if AttributeNotSet – call Field.get_if_attribute_not_set;
- if NotLoaded – call Field.get_if_not_loaded;
- call Field.from_mongo;
- set __name__ and __parent__
- save to __cache__
Call Field.get_default;
If AttributeNotSet – call Field.get_if_attribute_not_set;
Return value.
-
__set__
(instance, value)¶ Set value to document.
- Call Field.prepare_value for cast value;
- Save in Document.__changed__;
- Call Field.set_parent_changed.
-
-
class
yadm.fields.base.
Field
(smart_null=False)¶ Base field for all database fields.
Parameters: smart_null (bool) – If it True, access to not exists fields return None instead AttributeError exception. You will not be able to distinguish null value from not exist. Use with care. -
descriptor_class
¶ Class of desctiptor for work with field
-
document_class
¶ Class of document. Set in
contribute_to_class()
.
-
name
¶ Name of field in document. Set in
contribute_to_class()
.
-
contribute_to_class
(document_class, name)¶ Add field for document_class.
Parameters: document_class (MetaDocument) – document class for add
-
copy
()¶ Return copy of field.
-
descriptor_class
alias of
FieldDescriptor
-
from_mongo
(document, value)¶ Convert mongo value to python value.
Parameters: - document (BaseDocument) – document
- value – mongo value
Returns: python value
-
get_default
(document)¶ Return default value.
-
get_fake
(document, faker, deep)¶ Return fake data for testing.
-
get_if_attribute_not_set
(document)¶ Call if key not exist in document.
-
get_if_not_loaded
(document)¶ Call if field data marked as not loaded.
-
prepare_value
(document, value)¶ The method is called when value is assigned for the attribute.
Parameters: - document (BaseDocument) – document
- value – raw value
Returns: prepared value
It must be accept value argument and return processed (e.g. casted) analog. Also it is called once for the default value.
-
to_mongo
(document, value)¶ Convert python value to mongo value.
Parameters: - document (BaseDocument) – document
- value – python value
Returns: mongo value
-
Simple fields¶
Fields for basic data types.
-
class
yadm.fields.simple.
BooleanField
(default=<class 'yadm.markers.AttributeNotSet'>, *, choices=None, **kwargs)¶ Field for boolean values.
-
type
¶ alias of
bool
-
-
class
yadm.fields.simple.
FloatField
(default=<class 'yadm.markers.AttributeNotSet'>, *, choices=None, **kwargs)¶ Field for float.
-
type
¶ alias of
float
-
-
class
yadm.fields.simple.
IntegerField
(default=<class 'yadm.markers.AttributeNotSet'>, *, choices=None, **kwargs)¶ Field for integer.
-
type
¶ alias of
int
-
-
class
yadm.fields.simple.
ObjectIdField
(default_gen=False)¶ Field for ObjectId.
Parameters: default_gen (bool) – generate default value if not set -
type
¶ alias of
ObjectId
-
Datetime field¶
-
class
yadm.fields.datetime.
DatetimeField
(*, auto_now=False, **kwargs)¶ Field for time stamp.
Parameters: auto_now (bool) – datetime.now as default (default: False)
Decimal field¶
Field for decimal numbers
This code save to MongoDB document:
-
class
yadm.fields.decimal.
DecimalField
(*, context=None, **kwargs)¶ Field for work with
decimal.Decimal
.Parameters: - context (decimal.Context) – context for decimal operations
(default: run
decimal.getcontext()
when need) - default (decimal.Decimal) –
TODO: context in copy()
-
context
¶ Context.
Returns: decimal.Context
for values
-
prepare_value
(document, value)¶ Cast value to
decimal.Decimal
.
- context (decimal.Context) – context for decimal operations
(default: run
Embedded documents fields¶
Work with embedded documents.
class EDoc(EmbeddedDocument):
i = fields.IntegerField()
class Doc(Document):
__collection__ = 'docs'
edoc = EmbeddedDocumentField(EDoc)
doc = Doc()
doc.edoc = EDoc()
doc.edoc.i = 13
db.insert(doc)
-
class
yadm.fields.embedded.
EmbeddedDocumentField
(embedded_document_class, *, auto_create=True, **kwargs)¶ Field for embedded objects.
Parameters: - embedded_document_class (EmbeddedDocument) – class for embedded document
- auto_create (bool) – automatic creation embedded document from access
-
copy
()¶ Return copy of field.
-
get_if_attribute_not_set
(document)¶ Call if key not exist in document.
If auto_create is True, create and return new embedded document. Else AttributeError is raised.
Reference field¶
Work with references.
class RDoc(Document):
i = fields.IntegerField
class Doc(Document):
rdoc = fields.ReferenceField(RDoc)
rdoc = RDoc()
rdoc.i = 13
db.insert(rdoc)
doc = Doc()
doc.rdoc = rdoc
db.insert(doc)
doc = db.get_queryset(Doc).find_one(doc.id) # reload doc
assert doc.rdoc.id == rdoc.id
assert doc.rdoc.i == 13
-
exception
yadm.fields.reference.
BrokenReference
¶ Raise if referrenced document is not found.
-
exception
yadm.fields.reference.
NotBindingToDatabase
¶ Raise if set ObjectId insted referenced document to new document, who not binded to database.
Containers fields¶
Base classes for containers.
-
class
yadm.fields.containers.
Container
(field, parent, value)¶ Base class for containers.
-
reload
()¶ Reload all object from database.
-
List fields¶
List of objects.
class Doc(Document):
__collection__ = 'docs'
integers = fields.ListField(fields.IntegerField)
doc = Doc()
doc.integers.append(1)
doc.integers.append(2)
assert doc.integers == [1, 2]
db.insert(doc)
doc = db.get_queryset(Doc).find_one(doc.id) # reload
doc.integers.append(3) # do not save
assert doc.integers == [1, 2, 3]
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.integers == [1, 2]
doc.integers.remove(2) # do not save too
assert doc.integers == [1]
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.integers == [1, 2]
doc.integers.push(3) # $push query
assert doc.integers == [1, 2, 3]
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.integers == [1, 2, 3]
doc.integers.pull(2) # $pull query
assert doc.integers == [1, 3]
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.integers == [1, 3]
-
class
yadm.fields.list.
List
(field, parent, value)¶ Container for list.
-
append
(item)¶ Append item to list.
Parameters: item – item for append This method does not save object!
-
insert
(index, item)¶ Append item to list.
Parameters: - index (int) –
- item – item for insert
This method does not save object!
-
pull
(query, reload=True)¶ Pull item from database.
Parameters: - query – query for $pull on this field
- reload (bool) – automatically reload all values from database
See $pull in MongoDB’s update.
-
push
(item, reload=True)¶ Push item directly to database.
Parameters: - item – item for $push
- reload (bool) – automatically reload all values from database
See $push in MongoDB’s update.
-
remove
(item)¶ Remove item from list.
Parameters: item – item for remove This method does not save object!
-
replace
(query, item, reload=True)¶ Replace list elements.
Parameters: - query – query for update. Keys of this query is relative.
- item – embedded document or dict
- reload (bool) – automatically reload all values from database
-
update
(query, values, reload=True)¶ Update fields in embedded documents.
Parameters: - query – query for update. Keys of this query is relative.
- values – dict of new values
- reload (bool) – automatically reload all values from database
-
Set field¶
Field with sets.
Similar as yadm.fields.list
.
-
class
yadm.fields.set.
Set
(field, parent, value)¶ Container for set.
-
add
(item)¶ Append item to set.
Parameters: item – item for add This method does not save object!
-
add_to_set
(item, reload=True)¶ Add item directly to database.
Parameters: - item – item for $addToSet
- reload (bool) – automatically reload all values from database
See $addToSet in MongoDB’s update.
-
discard
(item)¶ Remove item from the set if it is present.
Parameters: item – item for discard This method does not save object!
-
pull
(query, reload=True)¶ Pull item from database.
Parameters: - query – query for $pull on this field
- reload (bool) – automatically reload all values from database
See $pull in MongoDB’s update.
-
remove
(item)¶ Remove item from set.
Parameters: item – item for remove This method does not save object!
-
Map field¶
Map.
class Doc(Document):
__collection__ = 'docs'
map = fields.MapField(fields.IntegerField)
doc = Doc()
doc.map['a'] = 1
doc.map['b'] = 2
assert doc.map == {'a': 1, 'b': 2}
db.insert(doc)
doc = db.get_queryset(Doc).find_one(doc.id) # reload
doc.map['c'] = 3 # do not save
assert doc.map == {'a': 1, 'b': 2, 'c': 3}
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.map == {'a': 1, 'b': 2}
del doc.map['b'] # do not save too
assert doc.map == {'a': 1}
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.map == {'a': 1, 'b': 2}
doc.map.set('d', 3) # $set query
assert doc.map == {'a': 1, 'b': 2, 'c': 3}
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.map == {'a': 1, 'b': 2, 'c': 3}
doc.map.unset('d', 3) # $unset query
assert doc.map == {'a': 1, 'b': 2}
doc = db.get_queryset(Doc).find_one(doc.id) # reload
assert doc.map == {'a': 1, 'b': 2}
-
class
yadm.fields.map.
Map
(field, parent, value)¶ Map.
-
set
(key, value, reload=True)¶ Set key directly in database.
Parameters: - key – key
- value – value for $set
See $set in MongoDB’s set.
-
unset
(key, reload=True)¶ Unset key directly in database.
Parameters: key – key See $unset in MongoDB’s unset.
-
-
class
yadm.fields.map.
MapCustomKeysField
(item_field, key_factory, *, key_to_str=<class 'str'>, auto_create=True, **kwargs)¶ Field for maps with custom key type.
Parameters: - item_field (field) –
- key_factory (func) – function, who return thue key from raw string key
- key_to_str (func) –
- auto_create (bool) –
Geo fields¶
Fields for geo data
See: http://docs.mongodb.org/manual/applications/geospatial-indexes/
GeoJSON: http://geojson.org/geojson-spec.html
-
class
yadm.fields.geo.
Geo
¶ Base class for GeoJSON data.
-
class
yadm.fields.geo.
GeoCoordinates
¶ Base class for GeoJSON data with coordinates.
-
class
yadm.fields.geo.
GeoField
(types=[<class 'yadm.fields.geo.Point'>, <class 'yadm.fields.geo.MultiPoint'>], **kwargs)¶ Base field for GeoJSON objects.
-
class
yadm.fields.geo.
GeoOneTypeField
(**kwargs)¶ Base field for GeoJSON objects with one acceptable type.
-
class
yadm.fields.geo.
MultiPoint
(points)¶ Class for GeoJSON MultiPoint objects.
-
class
yadm.fields.geo.
MultiPointField
(**kwargs)¶ Field for MultiPoint.
-
type
¶ alias of
MultiPoint
-
-
class
yadm.fields.geo.
Point
(longitude, latitude)¶ Class for GeoJSON Point objects.