Re: making tvix production-ready

Re: making tvix production-ready

[Florian Klink]

Hey Aspen,

I think most of the things you were experiencing was due to no alignment
on how to accomplish a certain thing before doing the entire
implementation, and then hitting a roadblock when people actually saw
the entire code.

Regarding Tvix builder, I just landed the patchsets bringing in OCI
builders. It's currently still failing for reasons documented in
https://cl.tvl.fyi/c/depot/+/12560 - I'm discussing with interested
folks about how to design this.

I'm also trying to spend some more time in getting the docs in shape,
which could help address some of your frustration/confusion on the
[ca]store bits, and general architectural questions / plans.

I'd like to brainstorm on some of the other points you mentioned, as
well as writing down and structuring some of the "roadmap". I however
don't think email is the right medium for this, this needs something
higher bandwidth and lower response times.

I'll be at NixCon, in case you planned coming there too, and am
generally also okay with meeting virtually. Please reach out directly
for scheduling, no need to loop in the entire list for this.

Cheers,
Florian



On Sat, Sep 28, 2024 at 12:35:14PM GMT, root@gws•fyi wrote:
>
>hey all!
>
>irc is somewhat breaking down for me as a primary method of
>communication, so I figured I'd try email.
>
>up front: I am considering dedicating some more-serious effort over the
>coming several weeks to getting tvix "production ready", for some
>meaning of the term, and want to talk to y'all before i do so to avoid a
>situation like https://cl.tvl.fyi/c/depot/+/12184 where I spend a chunk
>of time working on stuff only to hit a wall in code review because it
>disagrees with what you're thinking about in terms of design /
>architecture.
>
>my top priorities for the goals I'd like to achieve are:
>
>a. a functioning tvix builder (oci, bwrap,  something else...) with the
>   ability to directly trigger builds (/ realise store paths) from a
>   top-level CLI (`tvix build`, `:b` in a repl, etc.)
>b. a functioning refscan implementation, so we can start building paths
>   in nixpkgs and seeing what breaks
>
>lower priority, but also important, are:
>
>c. figuring out the design of a garbage collector for tvix-eval, and how
>   that integrates with other optimizations for eval like pointer tagging
>   and parallel evaluation
>d. any eval perf work that doesn't interact too deeply with gc
>
>in terms of goal a:
>
>the tangle of blob, directory, and path info services is quite difficult
>to follow, and frankly I feel like there's a lot of unjustified
>complexity there, but I'm sure there's a method to that madness?
>regardless, I would love some guidance here on what's necessary to get
>this done (eg, is it just landing https://cl.tvl.fyi/c/depot/+/12526 and
>then iterating from there? is there something else necessary here I can
>help with?)
>
>tazjin: can we decide on a plan for triggering builds at the top-level
>that all of us are happy with, rather than just blocking
>https://cl.tvl.fyi/c/depot/+/12184 ? there's complexity here around
>forcing thunks that makes it feel much more like the right point in the
>tradeoff space is to allow requesting the evaluator to build drvs
>directly, but maybe you have different thoughts than me
>
>in terms of goal b:
>
>see above re complexity of the tangled mess of services. where should
>this live? mostly looking at flokli here to point me in the right
>direction.
>
>re goals c and d:
>
>tazjin, i would like to get on the same page between the three of us
>about plans here; i have a ton of energy to work on tvix and would like
>to be able to put it in a direction that's not going to hit a wall of
>"nope, not that way".
>
>-
>aspen

-- 
Florian Klink