Skip to content

WU0009: Modules as types

This writeup explores the possibility of removing the incl <source> syntax for a more functional approach to "includes" or modules as a whole.

Existing solution

The include mechanism currently copies and paste source code in the current scope or in an inner scope. We need to figure out solutions around namespaces and module pathing to be able to use them properly, otherwise they are extremely annoying to use and prone to errors - with similar issues to C, where the lack of namespacing and copy-pasting of code is causing immense pain.

Problems

  1. Lack of scoping at the time
  2. Need special handling
  3. Need special handling for inner functions/types
  4. Need special path resolution - not present with regular jinko code

Proposed solution

Including a module creates a magical type, with fields corresponding to each item present in the module.

Bikeshedding

  • source
  • mod
  • module
  • incl
  • with
  • import
  • jk

Issues

How should the "importing" machinery work?

Let's imagine the bikeshedding discussion is over and we have settled on source as the name of the function responsible for importing a module into the current scope. The original plan was for the source function to have the following signature:

func source(path: string) -> type;

This function would create a new, anonymous type and return it - it could then be aliased like so:

type foo = source("foo");

The issue with this is that we do not only want a type, but also an instance of that type: if the module foo defines a function bar, we want foo.bar to point to this specific function, not otherwise. There are multiple solutions to this:

  1. Keep source as a function returning a type
func source(path: string) -> type;
  • Instantiate the type upon receptioning it
  • The default parameters for each of the fields contain the proper module values, functions and types
type foo = source("foo");
where foo = foo(); // instantiation 

// shorthand
where foo = source("foo")();

This opens up some possibilities such as specifying a module's function without changing the module or the code refering to it, as long as they are the exact same types? Is that something we want?

// foo.jk
func bar() -> int { 14 }

// main.jk
func my_bar() -> int { 16 }

where foo = source("foo")(
    bar: my_bar
);

foo.bar() // returns 16

However, the syntax is unwieldy - should we offer a shorthand so that TypeName; and TypeName(); are similar in the case where TypeName contains default values for all of its fields?

This would allow the following:

where foo_base = source("foo");
where foo_overriden = source("foo")(
    bar: my_bar,
);
  1. Have source be generic and return an instance of its type parameter
func source[T](path: string) -> T;
  • The magical type generated is now fully opaque and cannot be aliased, or not trivially at least
  • No module overriding (but do we want this in the first place)
  • Simpler syntax

  • Issues with external users specializing the source function?

  • Have source be a type, which offers specializations for each module

type source[T];

type source[T: "foo"](bar: func() -> int); // magic generated by the compiler
  • Same issue as 1
  • Also requires the shorthand initialization for record types with defaults everywhere to be wieldy.

Solution 1 with the type initialization shorthand has the cleanest syntax while offering module item overriding, which might be something interesting. If we do not want that, then solution 1 remains the clearest IMO.

Since we have UFCS, it also means that this syntax will be valid:

type foo = "foo".source();

Having the source mechanism be a function also allows extra optional parameters, such as the path of the file to source:

In Rust:

#[path = "utils/bar.jk"]
mod foo;

becomes

type foo = source("foo", path = "utils/bar.jk");

Requires first class types

Let's say that we want to source the following utils module:

// foo.jk

type Left[L](inner: L);
type Right[R](inner: R);
type Either[L, R] = Left[L] | Right[R];

func is_left[L, R](value: Either[L, R]) -> bool {
    switch value {
        _: Left -> true,
        _ -> false,
    }
}

Accessing the is_left function with the proposed solution is quite easy - our magic type will have an is_left member of type func[L, R](Either[L, R]) -> bool; But how do we actually access the Either type? utils.Either would be nice, but it does mean that the generated utils type needs a field named Either which would be of type... type?

First solution: Type specialization crimes

  1. Add type specialization
  2. Generate a basic util[T = ()] type which contains all of the members of our module except for the types.
  3. Specialize util as each type within the module, so something like this
where utils = source("utils");

// becomes

type utils[T = ()](
    is_left: func[L, R](Either[L, R]) -> bool,
)

type utils[L, T: Left](inner: L);
type utils[R, T: Right](inner: R);
type utils[L, R, T: Either] = utils[Left][L] | utils[Right][R];

which we can use like so:

// main.jk

where value: utils[Either] = utils[Left](inner: 156);

utils.is_left(value);
  • Problems with this solution
    • Different syntax from module functions and module variables
    • Unwieldy
    • Ugly as sin
    • Requires type specialization - do we want that?

Really not a fan of this solution

Second solution: First class types

where utils = source("utils");

// becomes

type utils(
    Left: type,
    Right: type,
    Either: type,
    is_left: func[L, R](utils.Either[L, R]) -> bool,
);

which we can use like so:

// main.jk

where value: utils.Either = utils.Left(inner: 156);

utils.is_left(value);
  • Problems with this solution
    • Requires self referential types (utils.is_left is of type func[L, R](utils.Either[L, R]) -> bool)
    • Requires first class type support
    • How to instantiate a utils.Either when we only know it's a type and not more?
      • Runtime type information?
      • How to compile that?

Side notes

type type information

Sidenote: type types should probably have way more information, similarly to func types:

type utils(
    Left: type[L](L),
    Right: type[R](R),
    Either: type[L, R](utils.Left[L] | utils.Right[R]),
    is_left: func[L, R](utils.Either[L, R]) -> bool,
);

Module overriding

How to propagate the changes to the rest of the instance if we do override a type's default parameter? Taking the above example, what should happen in this case?

type MyEither[L, R](inner: L | R);

where utils = source("utils")(
    Either: MyEither,
);

This won't work, because the Either field is of type type[L, R](utils.Left[L] | utils.Right[R]) - so since MyEither does not have the proper field types, it fails to typecheck. So it'll work only if we redefine Left and Right as well.

type MyL[T](T);
type MyR[T](T);
type MyEither[L, R](inner: MyL[L] | MyR[R]);

where utils = source("utils")(
    Left: MyL,
    Right: MyR,
    Either: MyEither,
);

I'm thinking that this mostly would be useful for functions, or for some niche optimizations?

  1. Functions

Let's say you'd like to add logging to the standard's library hashmap because you want to understand exactly how your values are being inserted in your map

where map = source("map")(
    insert: insert_with_log,
);

func insert_with_log[K, V](map: HashMap[K, V], key: K, value: V) -> HashMap[K, V] {
    // memoized so it's not too costly
    where original_map = source("map");

    println("inserting key {key} with value {value} into the map!");

    original_map.insert(map, key, value)
}

where map = HashMap;
where map = map.insert("key", "value"); // calls `insert_with_log`
  1. Niche optimizations

If you know that you will be using a small hashmap for example, you might want to override the map's module Vector type to use a SmallVec kind of implementation

type SmolVector = source("small_vec").SmallVec;

where map = source("map")(
    Vector: SmolVector,
);

This syntax needs a loooot of bikeshedding however. This is not pretty.

UFCS method resolution

How to decide what function 12.foo() should resolve to if foo was imported from a module?

  1. Enforce function reexporting
where module = source("bar");
where foo = module.foo;

12.foo(); // resolves to foo -> resolves to module.foo
  1. Look through the fields of variables in scope for a matching signature?
where module = source("bar");

12.foo(); // resolves to module.foo
  • Isn't that super unexpected as a behavior?
type SomeOtherType(
    foo: func(int),
);

func my_foo(a: int) {}

where some_other_thing = SomeOtherType(foo: my_foo);
where module = source("bar");

12.foo(); // resolves to module.foo OR some_other_thing.foo
// ambiguity? weird error reporting?