MServe

MServe is a RESTFul Web Framework for Service Providers. Its purpose is to provide human and machine usable interfaces to control the ingest, access, processing and manipulation of content using compute resources.

There are 3 main interfaces in MServe:

  • HTML (web browser) interface for human manipulation of the content;
  • HTTP/REST interface for machine workflows and automated systems;
  • WebDAV interface to provide file system access to the content.

MServe can host services for different users or customers each configured according to their needs with specific workflows for ingest, updates, access and periodic work created in a graphical interface.  Data storage is achieved through file-system mounts and there are built in processes for integrity checking and file replication and repair.  The software can handle large professional files such as MXF files in excess of 100GB.  Data processing is performed using the Celery distributed task queue along with RabbitMQ to distribute jobs on a cluster and third-party tools for tasks such as meta-data extraction and transcoding can easily be integrated.

A large number of measurements are made as the service executes, such as file sizes, retrieval times, jobs executed and successes or failures of e.g. SIP validation processes.  All this data is made available to the user and also can be sent to the Ting service management system for a global aggregated view and the potential for control via service level agreements.

Various standard technologies are employed to ensure robustness and ease integration, such as OAuth2 and OpenID for authentication, web-keys for easy sharing of data, and Apache, Django, Celery and RabbitMQ for the main framework.

MServe has its own website with full documentation and a demonstration service.  The source code is available on GitHub.