adpay.db package

This package provides database storage functionality.

The responsibility break down is:

• adpay.db provides database initialization and connection
• adpay.db.consts is the module hosting database configuration
• adpay.utils provides read and write tools to storage

adpay.db.utils module

class adpay.db.utils.QueryIterator(query)

Bases: object

Helper class for iterating over txmongo cursor queries.

Every query with cursor = True can be iterated with simple way:

_iter = query_iterator(query)

while True:

elem = yield _iter.next()

if elem is None:

break

print “elem”, elem

next(**kwargs)

Generator for items.

Returns:Yields a single value from cursor.

adpay.db.utils.delete_campaign(*args, **kwargs)

Remove campaign from the database.

Parameters:campaign_id – Main identifier. Returns:

adpay.db.utils.delete_campaign_banners(*args, **kwargs)

Remove banners (for a campaign) from the database.

Parameters:campaign_id – Campaign identifier. Returns:

adpay.db.utils.delete_event(*args, **kwargs)

Remove event from the database.

Parameters:event_id – Identifier of the event to remove. Returns:

adpay.db.utils.delete_keyword_frequency(*args, **kwargs)

Remove keyword frequency document.

Parameters:_id – Mongo document identifier. Returns:

adpay.db.utils.delete_payment_round(*args, **kwargs)

Remove the payment round from collection.

Parameters:timestamp – Returns:

adpay.db.utils.delete_user_keyword_frequency(*args, **kwargs)

Remove user keyword documents.

Parameters:_id – Mongo document _id. Returns:

adpay.db.utils.delete_user_profiles(*args, **kwargs)

Remove all user profile documents.

Returns:

adpay.db.utils.delete_user_scores(*args, **kwargs)

Parameters:

• campaign_id –
• timestamp –

Returns:

adpay.db.utils.get_banner(*args, **kwargs)

Fetch the banner document from the database.

Parameters:banner_id – Banner identifier. Returns:Banner document.

adpay.db.utils.get_banner_events_iter(*args, **kwargs)

Fetch all banner documents from the database.

Parameters:

• banner_id – Banner identifier.
• timestamp – Time in seconds since the epoch, used for getting the full hour timestamp.

Returns:

Iterable events collection (QueryIterator)

adpay.db.utils.get_banners_iter(*args, **kwargs)

Fetch all banner documents from the database.

Returns:Iterable banner collection (QueryIterator)

adpay.db.utils.get_campaign(*args, **kwargs)

Fetch the campaign document from the database.

Parameters:campaign_id – Campaign id Returns:Campaign document

adpay.db.utils.get_campaign_banners(*args, **kwargs)

Fetch the banner documents from the database for a given campaign.

Parameters:campaign_id – Campaign identifier. Returns:List of banner documents.

adpay.db.utils.get_campaign_iter(*args, **kwargs)

Fetch all campaign documents from the database.

Returns:Campaign iterable as a QueryIterator.

adpay.db.utils.get_distinct_users_from_events(*args, **kwargs)

Fetch distinct user identifiers for this campaign, for this timestamp.

Parameters:

• campaign_id – Campaign identifier.
• timestamp – Time in seconds since the epoch, used for getting the full hour timestamp.

Returns:

List of distinct users ids.

adpay.db.utils.get_events_per_user_iter(*args, **kwargs)

Fetch all events for this campaign, for this timestamp, for this uid.

Parameters:

• campaign_id – Campaign identifier.
• timestamp – Time in seconds since the epoch, used for getting the full hour timestamp.
• uid – User identifier.

Returns:

Iterable events for the user.

adpay.db.utils.get_keyword_frequency(*args, **kwargs)

Parameters:keyword – Returns:One keyword frequency document.

adpay.db.utils.get_no_updated_keyword_frequency_iter(*args, **kwargs)

Returns:Iterable of not updated keyword frequencies.

adpay.db.utils.get_payment_round(*args, **kwargs)

Fetch a payment round document from the database.

Parameters:timestamp – Timestamp (epoch, seconds, full hour) for this request. Returns:Payment round document.

adpay.db.utils.get_payment_round_iter(*args, **kwargs)

Returns:Iterable payment round collection.

adpay.db.utils.get_payments_iter(*args, **kwargs)

Fetch all payment documents from the database.

Parameters:timestamp – Timestamp (epoch, seconds, full hour) for this request. Returns:Iterable payment information.

adpay.db.utils.get_sorted_user_score_iter(*args, **kwargs)

Parameters:

• campaign_id –
• timestamp –
• limit –

Returns:

Descending by score sorted list of user score to limit.

adpay.db.utils.get_user_keyword_frequency(*args, **kwargs)

Parameters:

• user_id –
• keyword –

Returns:

One use keyword frequency document.

adpay.db.utils.get_user_keyword_frequency_distinct_userids(*args, **kwargs)

Returns:Distinct user ids.

adpay.db.utils.get_user_keyword_frequency_iter(*args, **kwargs)

Parameters:user_id – Returns:Iterable user keyword frequency collection.

adpay.db.utils.get_user_profile(*args, **kwargs)

Parameters:user_id – Returns:One user profile document.

adpay.db.utils.get_user_value_in_campaign(*args, **kwargs)

Parameters:

• campaign_id –
• user_id –

Returns:

One user value document.

adpay.db.utils.get_user_value_iter(*args, **kwargs)

Parameters:campaign_id – Returns:Iterable user value collection.

adpay.db.utils.set_keyword_frequency_updated_flag(*args, **kwargs)

Set/update the ‘updated’ flag in keyword frequency document.

Parameters:updated – Flag value (True/False) Returns:

adpay.db.utils.set_user_keyword_frequency_updated_flag(*args, **kwargs)

Set/update the ‘updated’ flag in user keyword frequency document.

Parameters:updated – Flag value (True/False) Returns:

adpay.db.utils.update_banner(*args, **kwargs)

Update banner data or create a new one if doesn’t exist.

Parameters:banner_doc – New banner data, must include banner_id. Returns:deferred instance of pymongo.results.UpdateResult.

adpay.db.utils.update_campaign(*args, **kwargs)

Update campaign data or create one if doesn’t exist.

Parameters:campaign_doc – New campaign data, must include campaign_id to identify existing data. Returns:deferred instance of pymongo.results.UpdateResult.

adpay.db.utils.update_event(*args, **kwargs)

Create or update an event if it doesn’t exist.

Parameters:event_doc – Event document Returns:

adpay.db.utils.update_event_payment(*args, **kwargs)

Create or update payment information for event.

Parameters:

• campaign_id – Campaign identifier.
• timestamp – Timestamp (epoch, in seconds, full hour)
• event_id – Event identifier
• payment – Payment amount.
• reason – Reason (payment classifier).

Returns:

adpay.db.utils.update_keyword_frequency(*args, **kwargs)

Create or update keyword frequency.

Parameters:

• keyword –
• frequency –
• updated –

Returns:

adpay.db.utils.update_payment_round(*args, **kwargs)

Create or update a payment round.

Parameters:timestamp – Returns:

adpay.db.utils.update_user_keyword_frequency(*args, **kwargs)

Create or update user keyword frequency document.

Parameters:

• user_id –
• keyword –
• frequency –
• updated –

Returns:

adpay.db.utils.update_user_profile(*args, **kwargs)

Create or update user profile.

Parameters:

• user_id – User identifier.
• profile_dict – Dicitonary of keyword scores, e.g. {‘keyword1’:score1, ‘keyword2’:score2, …}

Returns:

adpay.db.utils.update_user_score(*args, **kwargs)

Update user score with new score and timestamp, per campaign.

Parameters:

• campaign_id –
• timestamp –
• user_id –
• score –

Returns:

adpay.db.utils.update_user_value_in_campaign(*args, **kwargs)

Create or update the user value document.

Parameters:

• campaign_id –
• user_id –
• payment –
• human_score –

Returns:

Module contents

adpay.db.MONGO_CONNECTION = None

Global MongoDB connection

adpay.db.configure_db(*args, **kwargs)

Initialize the database, configure indexes.

Add a dummy campaigns for events without campaigns.

Returns:

adpay.db.disconnect(*args, **kwargs)

Disconnect the global connection

Returns:None

adpay.db.get_banner_collection(*args, **kwargs)

Return collection for banners.

Returns:Banner collection

adpay.db.get_campaign_collection(*args, **kwargs)

Return collection for campaigns.

Returns:Campaign collection

adpay.db.get_event_collection(*args, **kwargs)

Return collection for events.

Returns:Event collection

adpay.db.get_keyword_frequency_collection(*args, **kwargs)

Return collection for keyword frequencies (legacy).

Returns:Keyword frequency collection

adpay.db.get_mongo_connection(*args, **kwargs)

Returns:Global MongoDB connection (adpay.db.MONGO_CONNECTION)

adpay.db.get_mongo_db(*args, **kwargs)

Return specific, named database for this application.

Returns:Database

adpay.db.get_payment_collection(*args, **kwargs)

Return collection for payments.

Returns:Payments collection

adpay.db.get_payment_rounds_collection(*args, **kwargs)

Return collection for payment rounds.

Returns:Payment rounds collection

adpay.db.get_user_keyword_frequency_collection(*args, **kwargs)

Return collection for keywords frequencies in user profiles (legacy).

Returns:User keyword frequency collection

adpay.db.get_user_profile_collection(*args, **kwargs)

Return collection for user profiles (legacy).

Returns:User profile collection

adpay.db.get_user_score_collection(*args, **kwargs)

Return collection for user scores (legacy).

Returns:User score collection

adpay.db.get_user_value_collection(*args, **kwargs)

Return collection for user values (legacy).

Returns:User values collection