cliche.celery — Celery-backed task queue worker¶

Sometimes web app should provide time-consuming features that cannot immediately respond to user (and we define “immediately” as “shorter than a second or two seconds” in here). Such things should be queued and then processed by background workers. Celery does that in natural way.

We use this at serveral points like resampling images to make thumbnails, or crawling ontology data from other services. Such tasks are definitely cannot “immediately” respond.

@celery.task(ignore_result=True)
def do_heavy_work(some, inputs):
    '''Do something heavy work.'''
    ...

do_heavy_work.delay('some', inputs='...')

$ celery worker -A cliche --config dev.cfg.yml
 -------------- celery@localhost v3.1.13 (Cipater)
---- **** -----
--- * ***  * -- Darwin-13.3.0-x86_64-i386-64bit
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         cliche.celery:0x1... (cliche.celery.Loader)
- ** ---------- .> transport:   redis://localhost:6379/5
- ** ---------- .> results:     disabled
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[2014-09-12 00:31:25,150: WARNING/MainProcess] celery@localhost ready.

cliche.cli — Command-line interfaces¶

cliche.cli.initialize_app(config=None)¶

Initialize celery/flask app.

Parameters:	config – a config file path. accept .py, .yml file. default value is None

cliche.cli.config(func)¶

Provide --config or -c option and run initialize_app() automatically.

Parameters:	func (collections.abc.Callable) – a command function to decorate
Returns:	decorated func

cliche.cli.main = <click.core.Group object>¶

(collections.abc.Callable) The CLI entry point.

cliche.config — Reading configurations¶

class cliche.config.ConfigDict¶

Almost the same to the built-in dict except it raises ConfigKeyError instead of KeyError with finer error message.

exception cliche.config.ConfigKeyError¶

The exception raised when there’s no such configured key, that is a subtype of built-in KeyError.

cliche.config.read_config(filename)¶

Read Cliche app configuration from the given filename.

config = read_config(filename='dev.cfg.yml')

Note that it takes only one keyword argument at a time. All parameters are mutually exclusive for each other.

Parameters:	filename (pathlib.Path) – read config from a filename of yaml or python source code
Returns:	the parsed dictionary with uppercase keys
Return type:	ConfigDict

cliche.config.read_config_from_python(*, string=None, file=None, filename=None)¶

Read Cliche app configuration from Python code i.e. Flask-style configuration:

config = read_config_from_python(filename='dev.cfg.py')

Note that it takes only one keyword argument at a time. All parameters are mutually exclusive for each other.

Parameters:	string (str) – read config from a python source code string file – read config from a file object of python source code filename (pathlib.Path) – read config from a filename of python source code
Returns:	the parsed dictionary with uppercase keys
Return type:	collections.abc.Mapping

cliche.config.read_config_from_yaml(*, string=None, file=None, filename=None)¶

Read Cliche app configuration from YAML.

config = read_config_from_yaml(filename='dev.cfg.yml')

Note that it takes only one keyword argument at a time. All parameters are mutually exclusive for each other.

Parameters:	string (str) – read config from a yaml string file – read config from a file object of yaml filename (pathlib.Path) – read config from a filename of yaml
Returns:	the parsed dictionary with uppercase keys
Return type:	ConfigDict

cliche.credentials — Authentication methods¶

class cliche.credentials.TwitterCredential(**kwargs)¶

Information about Twitter User

id¶

(int) The primary key from Credential.id.

identifier¶

(int) Twitter user id

token¶

(str) The oauth token.

token_secret¶

(str) The oauth secret token.

cliche.orm — Object-relational mapping¶

Cliche uses the relational database and data on the database are mapped to objects. It widely uses SQLAlchemy as its ORM (object-relational mapping) framework.

In order to define a persist model class, just subclass Base:

from sqlalchemy import Column, Integer, UnicodeText

from .orm import Base


class Thing(Base):
    '''A something object-relationally mapped.'''

    id = Column(Integer, primary_key=True)
    value = Column(UnicodeText, nullable=False)
    __repr_columns = id, value
    __tablename__ = 'things'

class cliche.orm.Base(**kwargs)¶

SQLAlchemy declarative base class.

__repr_columns__¶

(collections.abc.Sequence) This columns will be printed to repr() string of its instances if __repr_columns__ is defined.

See also

SQLAlchemy — Declarative

Declarative allows all three to be expressed at once within the class declaration.

cliche.orm.Session = sessionmaker(class_='Session',autocommit=True, expire_on_commit=True, bind=None, autoflush=True)¶

SQLAlchemy session class.

See also

SQLAlchemy — Using the Session

Session is the primary usage interface for persistence operations.

cliche.orm.downgrade_database(engine, revision)¶

Reverts to a previous revision.

Parameters:	engine (sqlalchemy.engine.base.Engine) – the database engine to revert revision (str) – the previous revision to revert to

cliche.orm.get_alembic_config(engine)¶

Creates a configuration for alembic. You can pass an Engine object or a string of database url either. So:

from sqlalchemy import create_engine
engine = create_engine('postgresql://localhost/cliche')
alembic_cfg = get_alembic_config(engine)

is equivalent to:

db_url = 'postgresql://localhost/cliche'
alembic_cfg = get_alembic_config(db_url)

Parameters:	engine (sqlalchemy.engine.base.Engine, str) – the database engine to use
Returns:	an alembic config
Return type:	alembic.config.Config

cliche.orm.get_database_revision(engine)¶

Gets the current revision of the database.

Parameters:	engine (sqlalchemy.engine.base.Engine) – the database engine to get the current revision
Returns:	the script of the current revision
Return type:	alembic.script.Script

cliche.orm.import_all_modules(dry_run=False)¶

Import all submodules of cliche to ensure every ORM entity classes are ready to use. It’s useful for being ready to auto-generate a migration script.

Returns:	the set of module names
Return type:	collections.abc.Set

cliche.orm.initialize_database(engine)¶

Creates all database schemas and stamps it as the head of versions.

Parameters:	engine (sqlalchemy.engine.base.Engine) – the database engine to initialize

cliche.orm.upgrade_database(engine, revision='head')¶

Upgrades the database schema to the chosen revision (default is head).

Parameters:	engine (sqlalchemy.engine.base.Engine) – the database engine to upgrade revision (str) – the revision to upgrade to. default is 'head'

cliche.people — Artists, teams, and editors¶

class cliche.people.Person(**kwargs)¶

People i.e. artists, editors.

created_at¶

(datetime.datetime) The created time.

credits¶

(collections.abc.MutableSet) The set of cliche.work.Credits that the person has.

dob¶

(datetime.date) The date of birth.

dod¶

(datetime.date) The date of death.

id¶

(int) The primary key integer.

memberships¶

(collections.abc.MutableSet) The set of TeamMemberships he/she has.

teams¶

(collections.abc.MutableSet) The set of Teams he/she belongs to.

class cliche.people.Team(**kwargs)¶

Teams (including ad-hoc teams).

created_at¶

(datetime.datetime) The created time.

credits¶

(collections.abc.MutableSet) The set of cliche.work.Credits in which the team was involved.

id¶

(int) The primary key integer.

members¶

(collections.abc.MutableSet) The members Person set.

memberships¶

(collections.abc.MutableSet) The set of TeamMemberships that the team has.

class cliche.people.TeamMembership(**kwargs)¶

Team memberships to people.

created_at¶

(datetime.datetime) The added time.

member¶

(Person) The member who is a member of the team.

member_id¶

(int) Person.id of member.

team¶

(Team) The team that the member belongs to.

team_id¶

(int) Team.id of team.

cliche.services — Interfacing external services¶

from .crawler import crawl as sync  # noqa


__all__ = 'sync',

cliche.sqltypes — Collection of custom types for SQLAlchemy.¶

class cliche.sqltypes.EnumType(enum_class: enum.Enum, **kw)¶

Custom enum type to be used as enum.Enum`in Python standard library. It inherits :class:`sqlalchemy.types.SchemaType since it requires schema-level DDL. PostgreSQL ENUM type defined in an Alembic script must be explicitly created/dropped.

impl¶

alias of Enum

class cliche.sqltypes.HashableLocale(language, territory=None, script=None, variant=None)¶

Hashable Locale

class cliche.sqltypes.LocaleType(*args, **kwargs)¶

Custom locale type to be used as babel.Locale.

impl¶

alias of String

class cliche.sqltypes.UuidType(*args, **kwargs)¶

Custom UUID type to be used as uuid.UUID.

impl¶

alias of CHAR

cliche.user — Users¶

class cliche.user.User(**kwargs)¶

Registered user in cliche.io with social providers.

created_at¶

(datetime.datetime) The date and time on which the record was created.

credentials¶

(collections.abc.MutableSet) The credentials matched.

id¶

(int) The primary key integer.

name¶

(str) The name for using in user list.

cliche.web — Web application¶

cliche.work — Things associated with a creative work.¶

class cliche.work.Character(**kwargs)¶

Fictional character that appears in creative work.

derived_characters¶

(:class:’collections.abc.MutableSet’) The set of Characters which is derived from this character

id¶

(int) The primary key integer.

original_character¶

(:class:’Character’) The original character from which this character is derived.

original_character_id¶

(:class:’int’) Character.id of original_character.

works¶

(collections.abc.MutableSet) The set of Works in which the character appeared.

class cliche.work.Credit(**kwargs)¶

Relationship between the work, the person, and the team. Describe that the person participated in making the work.

created_at¶

(datetime.datetime) The date and time on which the record was created.

person¶

(cliche.people.Person) The person who made the work.

person_id¶

(int) cliche.people.Person.id of person.

role¶

The person’s role in making the work.

team¶

The team which the person belonged when work had been made.

team_id¶

(int) Team.id of team. (optional)

work¶

(Work) The work which the person made.

work_id¶

(int) Work.id of work.

class cliche.work.Franchise(**kwargs)¶

Multimedia franchise that is a franchise for which installments exist in multiple forms of media, such as books, comic books, and films, for example The Lord of the Rings and Iron Man.

created_at¶

(datetime.datetime) The date and time on which the record was created.

id¶

(int) The primary key integer.

work_franchises¶

(collections.abc.MutableSet) The set of WorkFranchises that the franchise has.

works¶

(collections.abc.MutableSet) The set of Works that belongs to the franchise.

world¶

(World) The world which the franchise belongs to.

world_id¶

(int) World.id of world.

class cliche.work.Genre(**kwargs)¶

Genre of the creative work

created_at¶

(datetime.datetime) The date and time on which the record was created.

id¶

(int) The primary key integer.

name¶

(str) The name of the genre.

work_genres¶

(collections.abc.MutableSet) The set of WorkGenres that the genre has.

works¶

(collections.abc.MutableSet) The set of Works that fall into the genre.

class cliche.work.Role¶

Python enum type to describe role of him/her in making a work.

class cliche.work.Work(**kwargs)¶

Creative work(s) that could be a single work like a film, or a series of works such as a combic book series and a television series.

characters¶

(collections.abc.MutableSet) The set of Characters that appeared in the work.

created_at¶

(datetime.datetime) The date and time on which the record was created.

credits¶

(collections.abc.MutableSet) The set of Credits that the work has.

franchises¶

(collections.abc.MutableSet) The set of Franchises that the work belongs to.

genres¶

(collections.abc.MutableSet) The set of Genres that the work falls into.

id¶

(int) The primary key integer.

media_type¶

(str) Work media type.

published_at¶

(datetime.date) The publication date.

tropes¶

(collections.abc.MutableSet) The set of Trope.

work_franchises¶

(collections.abc.MutableSet) The set of WorkFranchises that the work has.

work_genres¶

(collections.abc.MutableSet) The set of WorkGenres that the work has.

work_tropes¶

(collections.abc.MutableSet) The set of WorkTrope.

class cliche.work.WorkCharacter(**kwargs)¶

Relationship between the character and the work. Describe that the character appeared in the work.

character¶

(Character) The character that appeared in the work.

character_id¶

(int) Character.id of character.

created_at¶

(datetime.datetime) The date and time on which the record was created.

work¶

(Work) The work in which the character appeared.

work_id¶

(int) Work.id of work.

class cliche.work.WorkFranchise(**kwargs)¶

Relationship between the work and the Franchise.

created_at¶

(datetime.datetime) The date and time on which the record was created.

franchise¶

(Franchise) The franchise that the work belongs to.

franchise_id¶

(int) Franchise.id of franchise.

work¶

(Work) The work that belongs to the franchise.

work_id¶

(int) Work.id of work.

class cliche.work.WorkGenre(**kwargs)¶

Relationship between the work and the genre.

created_at¶

(datetime.datetime) The date and time on which the record was created.

genre¶

(Genre) The genre into which the work falls.

genre_id¶

(int) Genre.id of genre.

work¶

(Work) The work that falls into the genre.

work_id¶

(int) Work.id of work.

class cliche.work.World(**kwargs)¶

Fictional universe that is a self-consistent fictional setting with elements that differ from the real world, for example Middle-earth and Marvel Cinematic Universe.

created_at¶

(datetime.datetime) The date and time on which the record was created.

franchises¶

(collections.abc.MutableSet) The set of Franchises that belong the world.

id¶

(int) The primary key integer.