File Stores

A file store is responsible for:

1. reading and writing files (.mp3, .pdf, etc.) to/from an object store.
2. providing "transactions" that make all file operations atomic in a single request.
3. declaring the native object store client it delegates persistence operations to.

If a file store implementation is unable to handle a file InputStream, Athena pushes these responsibilities to the object store.

Included Stores

Athena comes bundled with a number of file stores:

1. Swift Store - A file store that can map operations on a file to an underlying OpenStack Swift API. Athena has explicit support for Swift
2. HDFS Store - File is persisted on Hadoop HDFS.

tip

It is assumed that the "HDFS Store" means a single-cluster HDFS. However, the Athena architecture does not preclude implementing a multi-cluster HDFS store

Stores can be included through the following artifact dependencies:

Swift Store

<dependency>
    <groupId>io.github.qubitpi.athena</groupId>
    <artifactId>athena-filestore-swift</artifactId>
    <version>${version.athena}</version>
</dependency>

HDFS Store

<dependency>
    <groupId>io.github.qubitpi.athena</groupId>
    <artifactId>athena-filestore-hdfs</artifactId>
    <version>${version.athena}</version>
</dependency>

Overriding the Store

To change the store, override the FileStore binding. For example, to use a store called SomeCustomFileStore:

public class AppBinderFactory extends AbstractBinderFactory {

    ...

    @Override
    protected Class<? extends FileStore> buildFileStore() {
        return SomeCustomFileStore.class;
    }
}

Custom Stores

Custom stores can be written by implementing the FileStore interface. Take Amazon S3 for instance

@Singleton
public class S3FileStore implements FileStore {

    private final AmazonS3 s3client;
    private final FileIdGenerator fileIdGenerator;

    @Inject
    public S3FileStore(@NotNull final AmazonS3 s3client, @NotNull final FileIdGenerator fileIdGenerator) {
        this.s3client = Objects.requireNonNull(s3client);
        this.fileIdGenerator = Objects.requireNonNull(fileIdGenerator);
    }

    @Override
    public String upload(final File file) {
        final String fileId = fileIdGenerator.apply(file);

        s3client.putObject(
                file.getMetaData().getFileType().name(),
                fileId,
                file.getFileContent(),
                new ObjectMetadata()
        );

        return fileId;
    }

    @Override
    public InputStream download(final String fileId) {
       return getS3client().getObject(...);
    }
}

Multiple Stores

A common pattern in Athena is the need to support multiple file stores. Typically, one file store manages most files, but some others may require a different object storage backend or have other needs to specialize the behavior of the store. The multiplex store in Athena manages multiple stores - delegating calls to the appropriate store which is responsible for a particular file.

This is a feature yet to be offered soon.