Chapter 11: First-Class Modules

Table of Contents

Working with First-Class Modules
Example: A Query-handling framework
Living without First-Class Modules

OCaml essentially has 2 parts to the language:

a core language that is concerned with values and types
- can’t contain modules or module types
  So we can’t define a variable whose value is a module or a function that takes a module as an argument.
a module language that is concerned with modules and module signatures
- can contain types and values

However, OCaml provides a language construct to circumvent this stratification: first-class modules – can be created from and converted back to regular modules

Letting modules into the core language is powerful, it increases the range of what we can express and makes it easier to build flexible and modular systems.

Working with First-Class Modules #

We’re going to use some toy examples to illustrate the following points:

Creating First-Class Modules (packaging) #

Creating first-class modules requires us to package a module up with a signature that satisfies it.

Given a module signature and a module definition, we have an example of how this packaging can be done using the syntax (module <Module> : <Module_type>)

(* Signature *)
open Base;;
module type X_int = sig val x : int end;;
(* module type X_int = sig val x : int end *)

(* Definition *)
module Three : X_int = struct let x = 3 end;;
(* module Three : X_int *)
Three.x;;
(* - : int = 3 *)

(* packaging it up at a first-class module *)
let three = (module Three : X_int);;
(* val three : (module X_int) = <module> *)

Inference and Anonymous Modules #

From the packaging syntax, we can module type if it can be inferred
We can also do this wrapping if the module is anonymous

(* inferrable *)
module Four = struct let x = 4 end;;
(* module Four : sig val x : int end *)
let numbers = [ three; (module Four) ];;
(* val numbers : (module X_int) list = [<module>; <module>] *)


(* anonymous *)
let numbers = [three; (module struct let x = 4 end)];;
(* val numbers : (module X_int) list = [<module>; <module>] *)

Unpacking first-class modules #

Since we know how to package a module into a first-class module, we should know how to unpack a module and access its contents using the syntax (val <first_class_module> : <Module_type>)

module New_three = (val three : X_int);;
(* module New_three : X_int *)
New_three.x;;
(* - : int = 3 *)

Functions for Manipulating First-Class Modules #

Since they are first-class we have some ways to manipulate them (including ordinary functions that consume and create first-class modules):

this function is about conversions between the first-class module (runtime) from the Module language (compile-time).

If we wish to do module-field projections, then it needs to be in Module language (unpacked) instead of the packed first-class language.

   (* consumes a module and returns the int within it *)
   let to_int m =
     let module M = (val m : X_int) in
     M.x;;
   (* val to_int : (module X_int) -> int = <fun> *)

   (* consumes 2 packed modules, returns a first-class module *)
   let plus m1 m2 =
     (module struct
        let x = to_int m1 + to_int m2
      end : X_int);;
   (* val plus : (module X_int) -> (module X_int) -> (module X_int) = <fun> *)

   let res = plus three (module Four);;
   module Res = (val res : X_int);;
   Res.x;; (* -- this is a field inspection*)

   (* this works *)
   module Foo = (val (plus three (module Four: X_int)) :  X_int);;
   Foo.x;; (* works because Foo is a module-identifier so Foo.x is a module-field projection. Projections work only for syntactic modules, not for runtime first-class values.*)

   (* this doesn't work:
      (*
      (val (plus three (module Four : X_int)) : X_int)
    *)
    this is not a module, it's a first-class value of a type -- an existential package. We need to unpack the first-class module (runtime) into a module binding (compile-time) then do the field projection.
    *)

   (* forcing out an inline example: *)
   let module Foo = (val (plus three (module Four : X_int)) : X_int) in
       Foo.x;;

We can pattern match to unpack a first-class module

   (* --- unpacking via pattern-match, more concise *)
   let to_int (module M : X_int) = M.x;;
   (* val to_int : (module X_int) -> int = <fun> *)


   (* OLD way: consumes a module and returns the int within it *)
   let to_int m =
     let module M = (val m : X_int) in
     M.x;;
   (* val to_int : (module X_int) -> int = <fun> *)

So all in all, we have good expressiveness:

   let six = plus three three;;
   (* val six : (module X_int) = <module> *)
   to_int (List.fold ~init:six ~f:plus [three;three]);;
   (* - : int = 12 *)

Richer First-Class Modules #

Let’s go beyond simple int values and let the first-class modules contain types and functions.

module type Bumpable = sig
  type t
  val bump : t -> t
end;;
(* module type Bumpable = sig type t val bump : t -> t end *)

(* multiple instances of this module signature: *)
module Int_bumper = struct
  type t = int
  let bump n = n + 1
end;;
(* module Int_bumper : sig type t = int val bump : t -> t end *)
module Float_bumper = struct
  type t = float
  let bump n = n +. 1.
end;;
(* module Float_bumper : sig type t = float val bump : t -> t end *)

(* we can package these as first-class modules: *)
let int_bumper = (module Int_bumper : Bumpable) and float_bumper = (module Float_bumper : Bumpable);;

Exposing Types #

Continuing on, int_bumper is fully abstract and so we can’t exploit the fact that the type in question is int. We can’t really do anything with values of Bumper.t.

we can’t do this:

let (module Bumper) = int_bumper in
Bumper.bump 3;;
(*
Line 2, characters 15-16:
Error: This expression has type int but an expression was expected of type
         Bumper.t
*)

To make int_bumper usable, we need to expose that the type Bumpable.t is equal to int for which we can use constraint sharing

let int_bumper = (module Int_bumper : Bumpable with type t = int);;
(* val int_bumper : (module Bumpable with type t = int) = <module> *)
let float_bumper = (module Float_bumper : Bumpable with type t = float);;
(* val float_bumper : (module Bumpable with type t = float) = <module> *)


(* usage works:*)
let (module Bumper) = int_bumper in
Bumper.bump 3;;
(* - : int = 4 *)
let (module Bumper) = float_bumper in
Bumper.bump 3.5;;
(* - : float = 4.5 *)

option 2: using a locally abstract type to make polymorphic first-class modules #

We can use first-class modules polymorphically, consider this function that takes in 2 args: Bumpable module and list of elements of the same type as type t of the module.

The type a (pseudoparameter) allows us to use a as a locally abstract type. a then acts like an abstract type within the context of the function.

In this example, we then use the locally abstract type as part of a sharing constraint that ties the type B.t with the type of the elements of the list passed in.

This makes the function polymorphic in both the type of the list element and the type Bumpable.t as we see in the usage section of the code example:

let bump_list
      (type a)
      (module Bumper : Bumpable with type t = a)
      (l: a list)
  =
  List.map ~f:Bumper.bump l;;

(*
val bump_list : (module Bumpable with type t = 'a) -> 'a list -> 'a list =
  <fun>
  *)
(* === polymorphic usage: *)
bump_list int_bumper [1;2;3];;
(* - : int list = [2; 3; 4] *)
bump_list float_bumper [1.5;2.5;3.5];;
(* - : float list = [2.5; 3.5; 4.5] *)

Polymorphic first-class modules are important because they allow you to connect the types associated with a first-class module to the types of other values you’re working with.

More on locally abstract types #

One of the key properties of locally abstract types is that they’re dealt with as abstract types in the function they’re defined within, but are polymorphic from the outside.

let wrap_in_list (type a) (x:a) = [x];;
(* val wrap_in_list : 'a -> 'a list = <fun> *) (*<-- "a" is used ina way that is compatible with it being abstract but hte type of the function that is inferred is polymorphic. *)

(* so compiler complains if we try this:  *)
let double_int (type a) (x:a) = x + x;;
(*
Line 1, characters 33-34:
Error: This expression has type a but an expression was expected of type int
*)

Locally abstract types are useful because they let us create a fresh type name that can be referenced inside type definitions:

can be used to build local modules
can be used to wire types to functors in a type-safe way

see how we create a new first-class module here:

module type Comparable = sig
  type t
  val compare : t -> t -> int
end;;
(* module type Comparable = sig type t val compare : t -> t -> int end *)
let create_comparable (type a) compare =
  (module struct
    type t = a (* this equality is internal to the module *)
    let compare = compare
  end : Comparable with type t = a);; (* -- the locally abstract type exposes the type equality: external to the module.*)
(*
val create_comparable :
  ('a -> 'a -> int) -> (module Comparable with type t = 'a) = <fun>
  *)
let int_compare = create_comparable Int.compare;;
(* - : (module Comparable with type t = int) = <module> *)
let float_compare = create_comparable Float.compare;;
(* - : (module Comparable with type t = float) = <module> *)
let module I_comp = (val int_compare) in (* seems like the module type is inferrable*)
    I_comp.compare 2 3;;

Disambiguating “abstract” and “polymorphic”

Concept	Meaning	Typical Context	Example
Abstract type	A type whose concrete representation is hidden or unknown.	Module boundaries or locally abstract types (inside their scope).	`type t` in a signature; `(type a)` inside `let`.

Polymorphic type	A function or value that works for any type.	`‘a`, `‘b` quantified type parameters.	`let id : ‘a -> ‘a = fun x -> x`.

Locally Abstract Type	Behaves as abstract inside its definition but is polymorphic outside.	GADTs, first-class modules, polymorphic recursion.	`let f (type a) (x : a expr) = …`

The difference is in their levels of generality and different mechanisms:

“Abstract” means “unknown (for now)”
- for an abstract type, the concrete representation is intentionally hidden
- it doesn’t mean that the type can vary between calls / doesn’t mean that it can be generalised across calls – it just means that we can’t see inside it
- compiler treats this like a distinct opaque name, unequal to others even if implementation is int or string underneath
  So type t is an abstract (but one fixed type), known only within M
```
     module M : sig
       type t           (* abstract -- to users of this module, t can't be treated as int *)
       val create : unit -> t
     end = struct
       type t = int     (* concrete inside the module *)
       let create () = 42
     end
```
“Polymorphic” means “general across types”
“Polymorphism” == “Generality across many possible types”.
so a polymorphic value or function will work uniformly over any type. Typically we define this using 'a instead of a.
```
   let id (x : 'a) : 'a = x
   (* id : 'a -> 'a *)
```
“locally abstract types” mix the two up:
“One of the key properties of locally abstract types is that they’re dealt with as abstract types in the function they’re defined within, but are polymorphic from the outside.”
When we say let f (type a) (x: a t) = ...:
1. inside the function f, we wish a to be treated as an abstract type – unknown, fixed and opaque (we can’t assume what a is)
  - inside the definition, a acts like an abstract placeholder (“opaque” – we only know “some type”)
2. from outside the function f, f is polymorphic in a: it can be used with any concrete instantiation of a.
  - so outside the definition, the function is universally quantified over a so we can call it with any concrete type (i.e. it’s polymorphic)

Example: A Query-handling framework #

We will have a system that responds to user-generated queries. System uses sexps for formatting queries and responses, and also the config for the query handler.

The signature here is for a module that implements a system for responding to user-generated queries. Other serialisation formats (JSON) could have worked too.

So here’s our Query_handler interface.

module type Query_handler = sig

  (** Configuration for a query handler *)
  type config

  val sexp_of_config : config -> Sexp.t (*serialises the config*)
  val config_of_sexp : Sexp.t -> config (*deserialised the serialised config*)

  (** The name of the query-handling service *)
  val name : string

  (** The state of the query handler *)
  type t

  (** Creates a new query handler from a config *)
  val create : config -> t

  (** Evaluate a given query, where both input and output are
      s-expressions *)
  val eval : t -> Sexp.t -> Sexp.t Or_error.t
end;;

sexp converters are tedious to implement by hand, we can use ppx_sexp_conv to generate the sexp converters based on their type definition.

We can also use the annotations within a signature to add the appropriate type signature. The purpose of applying the annotation to the interface is that the compiler knows that a module implementation that satisfies the signature has those sexp functions.

When applied within a type definition (implementation), the compiler will emit code for those functions – so code is generated.

  (* applying the annotation on a type to derive the sexp converter functions *)
  #require "ppx_jane";;
  type u = { a: int; b: float } [@@deriving sexp];;
  (*
  type u = { a : int; b : float; }
  val u_of_sexp : Sexp.t -> u = <fun>
  val sexp_of_u : u -> Sexp.t = <fun>
  *)
  sexp_of_u {a=3;b=7.};;
  (* - : Sexp.t = ((a 3)(b 7)) *)
  u_of_sexp (Core.Sexp.of_string "((a 43) (b 3.4))");;
  (* - : u = {a = 43; b = 3.4} *)

  (* using the annotation directly within the signature (implementation) *)
  module type M = sig type t [@@deriving sexp] end;;
  (*
  module type M =
    sig type t val t_of_sexp : Sexp.t -> t val sexp_of_t : t -> Sexp.t end
    *)

Same annotations can be attached within a signature to add the appropriate type signature:

Implementing a query handler #

Given the Query_handler interface, we can create some query handler modules.

module Unique = struct
  type config = int [@@deriving sexp]
  type t = { mutable next_id: int }

  let name = "unique"
  let create start_at = { next_id = start_at }

  let eval t sexp =
    (* NOTE: we expect the input for the query to be () which is Sexp.unit -- that's what the match expression is doing:*)
    match Or_error.try_with (fun () -> unit_of_sexp sexp) with
    | Error _ as err -> err
    | Ok () ->
      let response = Ok (Int.sexp_of_t t.next_id) in
      t.next_id <- t.next_id + 1;
      response
end;;

let unique = Unique.create 0;;
(* val unique : Unique.t = {Unique.next_id = 0} *)
Unique.eval unique (Sexp.List []);;
(* - : (Sexp.t, Error.t) result = Ok 0 *)
Unique.eval unique (Sexp.List []);;
(* - : (Sexp.t, Error.t) result = Ok 1 *)

Here’s another example of a query handler that does directory listings:

#require "core_unix.sys_unix";;

module List_dir = struct
  type config = string [@@deriving sexp] (* the default directory that relative paths are interpreted within *)
  type t = { cwd: string }

  (** [is_abs p] Returns true if [p] is an absolute path  *)
  let is_abs p =
    String.length p > 0 && Char.(=) p.[0] '/'

  let name = "ls"
  let create cwd = { cwd }

  let eval t sexp =
    match Or_error.try_with (fun () -> string_of_sexp sexp) with
    | Error _ as err -> err
    | Ok dir ->
      let dir =
        if is_abs dir then dir
        else Core.Filename.concat t.cwd dir
      in
      Ok (Array.sexp_of_t String.sexp_of_t (Sys_unix.readdir dir))
end;;


let list_dir = List_dir.create "/var";;
(* val list_dir : List_dir.t = {List_dir.cwd = "/var"} *)
List_dir.eval list_dir (sexp_of_string ".");;
(*
  - : (Sexp.t, Error.t) result =
Ok
 (yp networkd install empty ma mail spool jabberd vm msgs audit root lib db
  at log folders netboot run rpc tmp backups agentx rwho)
  *)

List_dir.eval list_dir (sexp_of_string "yp");;
(* - : (Sexp.t, Error.t) result = Ok (binding) *)

Dispatching to Multiple Query Handlers #

We’d like to create a whole dispatch table and dispatch queries.

It’s natural to use data structures like lists (or tables) using first-class modules, which would have been odd to do with modules and functors alone.

For the following example, assume (query-name query) is the shape of a single query, where query-name is the name used to determine which handler to dispatch the query to and query is the body of the query (as a sexp).

First we shall have a signature that combines a Query_handler module with an instantiated form of a query handler
- this part looks very similar to OOP code with the module type and the this
We can have a function that uses a locally abstract type to construct new instances, as a sort of a higher order function
then we an have a dispatch-table builder
then we can add in a dispatcher function
- this part looks like OOP code
  One key difference is that first-class modules allow you to package up more than just functions or methods. As we’ve seen, you can also include types and even modules. We’ve only used it in a small way here, but this extra power allows you to build more sophisticated components that involve multiple interdependent types and values.

(* === 1: signature to combine the Query_handler module with an instantiated form: *)
module type Query_handler_instance = sig
  module Query_handler : Query_handler
  val this : Query_handler.t
end;;
(*
module type Query_handler_instance =
  sig module Query_handler : Query_handler val this : Query_handler.t end
*)

(* === 2: using a locally abstract type, we can have an instance builder HOF *)
let build_instance
      (type a)
      (module Q : Query_handler with type config = a)
      config
  =
  (module struct
    module Query_handler = Q
    let this = Q.create config
  end : Query_handler_instance);;
(*
val build_instance :
  (module Query_handler with type config = 'a) ->
  'a -> (module Query_handler_instance) = <fun>
*)

(* -- this makes the construction of new instances all one-liners:  *)
let unique_instance = build_instance (module Unique) 0;;
(* val unique_instance : (module Query_handler_instance) = <module> *)
let list_dir_instance = build_instance (module List_dir)  "/var";;
(* val list_dir_instance : (module Query_handler_instance) = <module> *)


(* === 3: dispatch table builder: *)
let build_dispatch_table handlers =
  let table = Hashtbl.create (module String) in
  List.iter handlers
    ~f:(fun ((module I : Query_handler_instance) as instance) ->
      Hashtbl.set table ~key:I.Query_handler.name ~data:instance);
  table;;
(*
val build_dispatch_table :
  (module Query_handler_instance) list ->
  (string, (module Query_handler_instance)) Hashtbl.Poly.t = <fun>
  *)

(* === 4: dispatcher function to dispatch a query to a dispatch table *)
let dispatch dispatch_table name_and_query =
  match name_and_query with
  | Sexp.List [Sexp.Atom name; query] ->
    begin match Hashtbl.find dispatch_table name with
    | None ->
      Or_error.error "Could not find matching handler"
        name String.sexp_of_t
    | Some (module I : Query_handler_instance) -> (* NOTE: fn interacts with an instance by unpacking it into a module (I) and then using the query handler instance (I.this) in cert with the associated module (I.Query_handler) *)
      I.Query_handler.eval I.this query
    end
  | _ ->
    Or_error.error_string "malformed query";;
(*
val dispatch :
  (string, (module Query_handler_instance)) Hashtbl.Poly.t ->
  Sexp.t -> Sexp.t Or_error.t = <fun>
  *)

We can turn this into a CLI-interface code:

(* === 1: signature to combine the Query_handler module with an instantiated form: *)
module type Query_handler_instance = sig
  module Query_handler : Query_handler
  val this : Query_handler.t
end;;
(*
module type Query_handler_instance =
  sig module Query_handler : Query_handler val this : Query_handler.t end
*)

(* === 2: using a locally abstract type, we can have an instance builder HOF *)
let build_instance
      (type a)
      (module Q : Query_handler with type config = a)
      config
  =
  (module struct
    module Query_handler = Q
    let this = Q.create config
  end : Query_handler_instance);;
(*
val build_instance :
  (module Query_handler with type config = 'a) ->
  'a -> (module Query_handler_instance) = <fun>
*)

(* -- this makes the construction of new instances all one-liners:  *)
let unique_instance = build_instance (module Unique) 0;;
(* val unique_instance : (module Query_handler_instance) = <module> *)
let list_dir_instance = build_instance (module List_dir)  "/var";;
(* val list_dir_instance : (module Query_handler_instance) = <module> *)


(* === 3: dispatch table builder: *)
let build_dispatch_table handlers =
  let table = Hashtbl.create (module String) in
  List.iter handlers
    ~f:(fun ((module I : Query_handler_instance) as instance) ->
      Hashtbl.set table ~key:I.Query_handler.name ~data:instance);
  table;;
(*
val build_dispatch_table :
  (module Query_handler_instance) list ->
  (string, (module Query_handler_instance)) Hashtbl.Poly.t = <fun>
  *)

(* === 4: dispatcher function to dispatch a query to a dispatch table *)
let dispatch dispatch_table name_and_query =
  match name_and_query with
  | Sexp.List [Sexp.Atom name; query] ->
    begin match Hashtbl.find dispatch_table name with
    | None ->
      Or_error.error "Could not find matching handler"
        name String.sexp_of_t
    | Some (module I : Query_handler_instance) -> (* NOTE: fn interacts with an instance by unpacking it into a module (I) and then using the query handler instance (I.this) in cert with the associated module (I.Query_handler) *)
      I.Query_handler.eval I.this query
    end
  | _ ->
    Or_error.error_string "malformed query";;
(*
val dispatch :
  (string, (module Query_handler_instance)) Hashtbl.Poly.t ->
  Sexp.t -> Sexp.t Or_error.t = <fun>
  *)

open Stdio;;
let rec cli dispatch_table =
  printf ">>> %!";
  let result =
    match In_channel.(input_line stdin) with
    | None -> `Stop
    | Some line ->
      match Or_error.try_with (fun () ->
        Core.Sexp.of_string line)
      with
      | Error e -> `Continue (Error.to_string_hum e)
      | Ok (Sexp.Atom "quit") -> `Stop
      | Ok query ->
        begin match dispatch dispatch_table query with
        | Error e -> `Continue (Error.to_string_hum e)
        | Ok s    -> `Continue (Sexp.to_string_hum s)
        end;
  in
  match result with
  | `Stop -> ()
  | `Continue msg ->
    printf "%s\n%!" msg;
    cli dispatch_table;;
(*
val cli : (string, (module Query_handler_instance)) Hashtbl.Poly.t -> unit =
  <fun>
  *)




let () =
  cli (build_dispatch_table [unique_instance; list_dir_instance]);;

Loading and Unloading Query Handlers #

First-class modules give a lot of dynamism and flexibility – e.g. we can make it such that our query handlers can be loaded and unloaded at runtime.

We will define a Loader module:

define a Loader module that controls a set of active query handlers
define a creator function for creating a Loader.t
define functions (load, unload) to manipulate the table of active query handlers
define eval function which determines the query interface presented to the user
- first we create variant type for the request
- then we use the sexp converter generated for that type to parse the query from the user

(* ==== 1: loader module *)
module Loader = struct
  type config = (module Query_handler) list [@sexp.opaque]
  [@@deriving sexp]

  (** Loader.t has 2 tables: one for known modules and one for active handler instances *)
  type t = { known  : (module Query_handler)          String.Table.t
           ; active : (module Query_handler_instance) String.Table.t
           }

  let name = "loader"

(* ==== 2: function for creating a Loader.t  *)
let create known_list =
    let active = String.Table.create () in (* NOTE: the active table starts off as empty*)
    let known  = String.Table.create () in
    List.iter known_list
      ~f:(fun ((module Q : Query_handler) as q) ->
        Hashtbl.set known ~key:Q.name ~data:q);
    { known; active }

(* ==== 3: functions for manipulating the table of active query handlers.  *)
(* LOAD function: *)
let load t handler_name config =
    if Hashtbl.mem t.active handler_name then
      Or_error.error "Can't re-register an active handler"
        handler_name String.sexp_of_t
    else
      match Hashtbl.find t.known handler_name with
      | None ->
        Or_error.error "Unknown handler" handler_name String.sexp_of_t
      | Some (module Q : Query_handler) ->
        let instance = (* NOTE: instance is the first-class module that we create here*)
          (module struct
             module Query_handler = Q
             let this = Q.create (Q.config_of_sexp config)
           end : Query_handler_instance)
        in
        Hashtbl.set t.active ~key:handler_name ~data:instance;
        Ok Sexp.unit

(* UNLOAD FUNCTION *)
let unload t handler_name =
    if not (Hashtbl.mem t.active handler_name) then
      Or_error.error "Handler not active" handler_name String.sexp_of_t
    else if String.(=) handler_name name then
      Or_error.error_string "It's unwise to unload yourself"
    else (
      Hashtbl.remove t.active handler_name;
      Ok Sexp.unit
    )

(* ==== 4: request variant type and the eval function:  *)
type request =
    | Load of string * Sexp.t
    | Unload of string
    | Known_services
    | Active_services
  [@@deriving sexp]

let eval t sexp =
    match Or_error.try_with (fun () -> request_of_sexp sexp) with
    | Error _ as err -> err
    | Ok resp ->
      match resp with
      | Load (name,config) -> load   t name config
      | Unload name        -> unload t name
      | Known_services ->
        Ok ([%sexp_of: string list] (Hashtbl.keys t.known))
      | Active_services ->
        Ok ([%sexp_of: string list] (Hashtbl.keys t.active))
end
(* ^ ======= Loader module is complete now *)

Combining it with the Cli interface:

create an instance of the loader query handler
add that instance to the loader’s active table
then launch the cli interface, passing it the active table.

let () =
  let loader = Loader.create [(module Unique); (module List_dir)] in
  let loader_instance =
    (module struct
       module Query_handler = Loader
       let this = loader
     end : Query_handler_instance)
  in
  Hashtbl.set loader.Loader.active
    ~key:Loader.name ~data:loader_instance;
  cli loader.active

dynamic linking facilities #

KIV this but OCaml’s dynamic linking facilities allows us to compile and link in new code to a running program.

WE can automate this using libraries like ocaml_plugin which can be installed via OPAM and takes care of the workflow around setting up dynamic linking.

MAGIC: of OCaml #

MAGIC: The Magic of OCaml: Type Safety Meets Dynamic Linking

Seems like one of the most profound strengths of OCaml’s type system is how it extends safety and correctness all the way to the boundaries between separately compiled modules. This power shines when we consider dynamic linking — the ability to compile new code and load it into a running system.

Imagine a Mars rover millions of kilometers away that needs a software patch. You can’t afford runtime surprises.

In a dynamically typed language, replacing a module at runtime carries risk: the new code might not conform to the old module’s expectations until it actually fails in the field.

But in OCaml, the compiler enforces type integrity at every stage — even across separately compiled units. Each module’s interface carries a precise type signature, and the compiler ensures that any dynamically linked module matches that interface exactly. The runtime (Dynlink) can then safely load the compiled code (.cmxs files) knowing that the functions, values, and data layouts all align perfectly with the system’s expectations.

This means dynamic linking in OCaml isn’t an act of faith — it’s a provably safe operation. The strong static type system guarantees that what you load at runtime behaves exactly as the rest of the program expects.

So the “magic” of OCaml lies in this combination:

Static type safety gives compile-time guarantees about correctness and consistency.
Dynamic linking allows runtime adaptability.
Together, they enable systems that can evolve and patch themselves — even from millions of kilometers away — with mathematical confidence in their integrity.

Living without First-Class Modules #

Most designs that can be done with first-class modules can be simulated without them (though it’s a little awkward to do that).

The idea here is that we hide the true types of the objects in question behind the functions stored in the closure. We’ve implemented our query_handler_instance as just types below. As for Unique query handler into this framework, we just

type query_handler_instance =
  { name : string
  ; eval : Sexp.t -> Sexp.t Or_error.t };;
(*
type query_handler_instance = {
  name : string;
  eval : Sexp.t -> Sexp.t Or_error.t;
}
*)
type query_handler = Sexp.t -> query_handler_instance;;
(* type query_handler = Sexp.t -> query_handler_instance *)

(* ==== 2: putting the Unique query handler into this closure-based approach *)
let unique_handler config_sexp =
  let config = Unique.config_of_sexp config_sexp in
  let unique = Unique.create config in
  { name = Unique.name
  ; eval = (fun config -> Unique.eval unique config)
  };;

(* val unique_handler : Sexp.t -> query_handler_instance = <fun> *)

This is a small scale example and so it’s alright to not use first-class functions. When it gets a lot more complex (more functionality to be hidden away behind a set of closures, more complicated the r/s between the different types in question) then the more awkward this gets and it’s better to use first-class modules at that point.