castore = cast + store

A very minimal and simple library to interact with different storage systems

File systems and storage systems all expose very similar, but somewhat different, interfaces to interact with files and directories. Here we provide a very simple and basic interface that abstract various storage systems.

To use castore you can add the following maven dependency:

<dependency>
    <groupId>uk.ac.standrews.cs</groupId>
    <artifactId>castore</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>

This project is not available via a maven repository yet, so you will have to install this yourself by running mvn install.

Alternatively you can produce a .jar for the project, mvn package -am -DskipTests.

---

Available implementations

File Based

This implementation is based on the local file system.

Settings: type, root path

AWS.s3

In order to work with an AWS S3 storage, you need to provide an access_key_id and a secret_access_key. This can be done explicitly or by setting the environment variables:

export AWS_ACCESS_KEY_ID=<KEY>
export AWS_SECRET_KEY=<KEY>

Settings: type, bucket name, credentials (optional)

Redis

Redis is an in-memory data structure store, which is networked and very fast. A Redis-based storage can be used to store small amount of data with good performance.

Before using the Redis-based storage, you need Redis installed. Then run a server instance: redis-server

Settings: type, host, port (optional)

Dropbox

First you need to create a Dropbox app and get an App access token.

Settings: type, path to dropbox folder, access token

More on how to generate an access token here

Google Drive

Need to create a Google Drive App.

Settings: type, credentials json file

No need to set the path for the credentials file explicitly, if at the path “~/.drive/credentials.json”

Examples

Examples are available here.

---

Future Plans

• More implementations:
  • In-memory
  • GUID-based
  • OneDrive
  • WebDAV
  • FTP
• Make a storage instance available via a REST API
• CLI

---

Continous Integration

CircleCI automatically builds and runs the tests for the project.

Codecov, with JaCoCo, is used to measure the test coverage for the project. Info on how to read the Codecov reports: https://docs.codecov.io/v4.3.6/docs/viewing-source-code

CircleCI Settings

The following are the settings used in CircleCI to run the tests successfully.

Environment Variables

• DRIVE_SERVICE_KEY = result of base64 <your-service-account.json>
• DROPBOX_TOKEN = secret token generated by Dropbox
• AWS_ACCESS_KEY_ID and AWS_SECRET_KEY = as generated by AWS

Pre-tests

mkdir -p ${HOME}/.drive/
touch ${HOME}/.drive/credentials.json
echo ${DRIVE_SERVICE_KEY} | base64 --decode > ${HOME}/.drive/credentials.json

---

Credits

This project is inspired by the asa/filesystem project developed by Alan Dearle, Graham Kirby, and Stuart Norcross at the University of St Andrews.

The castore project has been developed, and is currently maintained, by Simone I. Conte at the University of St Andrews.

Suggestions and Pull Requests are more than welcome.