# Models

Models are standard Marcelle components with two additional characteristics. First, they have a property called parameters, which is a record of parameter values as streams. This structure is useful to provide interfaces that dynamically change the values of the model parameters. Second, they carry a set of methods for training and prediction. Some methods are standardized, such as .train(dataset) and .predict(features), however models can expose additional specific methods.

# Interface

Models implement the following interface:

interface Model<InputType, OutputType> {
  parameters: {
    [name: string]: Stream<any>;
  };

  $training: Stream<TrainingStatus>;

  train(dataset: Dataset): void;
  predict(x: InputType): Promise<OutputType>;

  save(update: boolean, metadata?: Record<string, unknown>): Promise<ObjectId | null>;
  load(id?: ObjectId): Promise<StoredModel>;
  download(metadata?: Record<string, unknown>): Promise<void>;
  upload(...files: File[]): Promise<StoredModel>;
  sync(name: string): void;
}

Models expose a $training stream that monitors the training process. Each TrainingStatus event has the following interface:

interface TrainingStatus {
  status: 'idle' | 'start' | 'epoch' | 'success' | 'error' | 'loaded' | 'loading';
  epoch?: number;
  epochs?: number;
  data?: Record<string, unknown>;
}

Where the data field varies across models to include additional information, such as the training and validation loss/accuracy in the case of neural networks.

# Common Methods

# .train()

train(dataset: Dataset): void;

Train the model from a given dataset.

# .predict()

async predict(x: InputType): Promise<OutputType>;

Make a prediction from a single input frame. Input and output formats vary across models, refer to each model's specific documentation below.

# .save()

save(update?: boolean, metadata?: Record<string, unknown>): Promise<ObjectId | null>;

Save the model to its associated datastore. The datastore can either be passed in the constructor's options (dataStore field), or by modifying the dataStore property of the model.

# Parameters

Option	Type	Description	Required	Default
update	boolean	If true, the model will try to update the dataStore's object to which it is associated, otherwise it will create a new document.		false'
metadata	object	A JSON-serializable object containing arbitrary model metadata		{}

# Returns

A promise that resolves with the ObjectId of the document in the data store, or null is the saving failed.

# .load()

load(id?: ObjectId): Promise<StoredModel>;

Load a model from its associated datastore. The datastore can either be passed in the constructor's options (dataStore field), or by modifying the dataStore property of the model.

# Parameters

Option	Type	Description	Required	Default
id	ObjectId	The ID of the model's document in the data store.

# Returns

A promise that resolves with the the StoredModel document from the data store, which has the following interface:

interface StoredModel {
  id?: ObjectId;
  name: string;
  url: string;
  metadata?: Record<string, unknown>;
}

# .download()

download(metadata?: Record<string, unknown>): Promise<void>;

Download a model as files, with optional custom metadata.

# Parameters

Option	Type	Description	Required	Default
metadata	object	A JSON-serializable object containing arbitrary model metadata		{}

# .upload()

upload(...files: File[]): Promise<StoredModel>;

Upload a model from a set of files. Files should be exported from Marcelle by the same model class.

# Parameters

Option	Type	Description	Required	Default
files	File[]	A list of files constituting the model (for instance, a `model.json` and a set of `.bin` weight files for a TFJS model)		{}

# .sync()

sync(name: string): void;

Synchronize a model with a data store, given a model name. The model will be automatically updated in the store whenever its training ends, or it is loaded from files. The model will be automatically restored on startup, from the latest version available in the store.

# Parameters

Option	Type	Description	Required	Default
name	string	A unique name for the model so that it can be retrieved in the datastore	yes

# BatchPrediction

marcelle.batchPrediction({ name: string, dataStore?: DataStore }): BatchPrediction;

This component allows to compute and store batch predictions with a given model over an entire dataset. Similarly to Datasets, the prediction results are stored in a data store passed in argument.

# Parameters

Option	Type	Description	Required
name	string	The name of the predictions (for data storage)	✓
dataStore	DataStore	The dataStore used to store the instances of the dataset. This parameter is optional. By default, a data store in memory will be created.

# Streams

Name	Type	Description	Hold
$predictions	Stream<ObjectId[]>	Stream of all the ids of the predictions stored in the data store	✓
$count	Stream<number>	Total number of predictions	✓

# Methods

# .predict()

async predict(model: Model, dataset: Dataset, inputField = 'features'): Promise<void>

Compute predictions for all instances of a given Datasets dataset, using a trained model. The instance field used for predictions can be specified with the inputField parameters, that defaults to features.

# .clear()

async clear(): Promise<void>

Clear all predictions from the data store, resetting the resulting streams.

# Example

const classifier = marcelle.mlp({ layers: [64, 32], epochs: 20 });

const batchMLP = marcelle.batchPrediction({ name: 'mlp', dataStore: store });

const predictButton = marcelle.button('Update predictions');
predictButton.$click.subscribe(async () => {
  await batchMLP.clear();
  await batchMLP.predict(classifier, trainingSet);
});

← Data Storage Utilities →