Wednesday , November 25 2020

liaopeiyuan / zeta, Hacker News



functional neural networks in ocaml

Features of zeta

1. Pedagogical

zeta does not particularly aim for performance, though I will make sure that reasonable demos are runnable. The source code of zeta is designed to be easy-to-read and succinct so that the user can get more beyond merely using this library for their daily research by reading them. I will later add more documentations and possibly tutorials for this library.

2. Functional

One of the most annoying error messages I’ve encountered in PyTorch looks something like this:

RuntimeError : expected Double tensor (got Float tensor)

zeta aims to "moves" errors like this from runtime to compile-time by adopting a functional programming paradigm in OCaml.

3. Dynamic Computation Graphs

zeta provides interfaces similar to that of the PyTorch, where users can create a computational graph on-the-fly.

4. Imperative

The implementation of zeta's core module, Tensor, is inherently imperative. This is to help create a more efficient representation of a computation graph, and therefore a neural network.

5. ADTs / GADTs (Algebraic Data Types / Generalized Algebraic Data Types)

One of the main contributions of zeta is to abstract neural network and tensor operations into numerous ADTs / GADTs, and in the process summarizing some of the basic behaviors deep learning algorithms exhibit. For example, a tensor can be recursively defined as a GADT:

type 'a tensordata=      | IntScalar: int ref ->int tensordata       | FloatScalar: float ref ->float tensordata       | BoolScalar: bool ref ->bool tensordata       | IntTensor: int tensordata array ->int tensordata       | FloatTensor: float tensordata array ->float tensordata       | BoolTensor: bool tensordata array ->bool tensordata

Which inherently restricts creations of ill-typed tensors, e.g., implicit casting is performed in this PyTorch example:

******************>>b=torch.FloatTensor ([False])>>>b tensor ([0.])

But the following would not type check in zeta:

let a=FloatTensor [| BoolScalar (ref false) |] ;; Error: This expression has type bool tensordata        but an expression was expected of type float tensordata        Type bool is not compatible with type float

Tensor viewing, slicing, reshaping, concatenating

About admin

Leave a Reply

Your email address will not be published. Required fields are marked *