Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Big interpreter rewrite #5406

Draft
wants to merge 60 commits into
base: main
Choose a base branch
from

Conversation

StachuDotNet
Copy link
Member

@StachuDotNet StachuDotNet commented Jul 20, 2024

Rewriting RuntimeTypes, the Interpreter, and PT2RT to:

Here are all of the wins (WIP):

  • much better, structured RTEs
    • OldStringErrorTODO is gone
  • better handling of RTEs, generally
    • especially dev experience (understanding what went wrong)
  • async, concurrent function calls
  • no easy-to-reach stack-overflows
  • more tests
  • super-fast builtins (without copying Dvals all over the place)
  • non-recursive runtime type-checking is now complete and I can close Refactor typechecking to use value types #5071 Use package Bool.and and Bool.or #5030
  • everything's faster
  • hopefully fast enough for 95% of programming tasks
  • partial-application works, for "applys" to both fns and lambdas
  • I might add some other tiny language features? lol
  • interpreter is easier to understand
  • there might end up actually being a net negative in terms of LOC at the end of this?
  • we're a tiny bit closer to getting tracing working in this new world
  • I have some ideas on further worthwhile optimizations (will hand off to Ocean)

I might get stuck on SqlCompiler and circle back to that in another PR - not sure yet.

It'll probably be another 2 weeks until this gets merged.

This closes #5222, closes #5130, closes #5071

@StachuDotNet StachuDotNet force-pushed the big-interpreter-rewrite branch 3 times, most recently from 21af1ef to 8a3496c Compare July 31, 2024 19:51
@StachuDotNet StachuDotNet force-pushed the big-interpreter-rewrite branch 3 times, most recently from 4abbe16 to 29faccd Compare August 13, 2024 13:57
@StachuDotNet
Copy link
Member Author

StachuDotNet commented Aug 14, 2024

thoughts on implemented match that I wanted to record in case my current plan doesn't end up working out: https://gist.github.com/StachuDotNet/5a8923721b255ad2a91c1c6dcc3e9b8e

Edit: it worked out (really well!) but might as well keep this around.

@StachuDotNet StachuDotNet force-pushed the big-interpreter-rewrite branch 2 times, most recently from 2c4303b to 6ae7196 Compare August 15, 2024 16:13
@StachuDotNet StachuDotNet force-pushed the big-interpreter-rewrite branch 4 times, most recently from 713821d to 4299937 Compare September 18, 2024 15:07
Comment on lines +188 to +193
(match Map.tryFind (parentContext, lambdaID) vm.lambdaInstrCache with
| Some l -> l
| None ->
match Map.tryFind (Source, lambdaID) vm.lambdaInstrCache with
| Some l -> l
| None -> raiseRTE (RTE.VariableNotFound "lambda not found")) // TODO better error
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, why do we need this fallback case? (Source) - When is that code hit?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is for the cases that we were trying to fix for -- but will the solution work if there are multiple nested lambdas? Or if a lambda is returned by a fn and used later on?

Copy link
Collaborator

@OceanOak OceanOak Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is hit when running functions that take a lambda as an argument (e.g. List.takeWhile, List.sortByComparator, List.sortBy, List.member, List.map, etc.)

The lambda couldn't be found in the currentFrame.context which is a PackageFn, instead it is found in the Source, I wasn't sure if it was the best solution.

As for "will the solution work if there are multiple nested lambdas" it works for some functions and doesn't for others (I am encountering other errors in these cases that don't seem to be related to this issue)

Comment on lines +434 to +436
vm.lambdaInstrCache
|> Map.add (currentFrame.context, impl.exprId) impl
|> Map.add (Source, impl.exprId) impl
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is a similar question - why do we need to add two entries?

Copy link
Collaborator

@OceanOak OceanOak Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the lambda is created in the Source context, but we try to apply it in the lambda that is inside a PackageFn context, here is an example of one of those functions

let map (list: List<'a>) (fn: 'a -> 'b) : List<'b> =
  (Stdlib.List.fold list [] (fun acc elem -> Stdlib.List.push_v0 acc (fn elem)))
  |> Stdlib.List.reverse

I've added this to make sure that we can find the lambda later regardless of the context in which it's being applied. Again not sure if it is the right thing to do, please let me know if you think there is a better way to do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: List.map Causes Stack Overflow Error on Large Lists
2 participants