Skip to content

Update *.witx files for the witx 0.9 syntax #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Feb 18, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/Proposals.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,11 @@ Proposals follow [this process document](https://github.com/WebAssembly/WASI/blo
| ------------------------------------------------------------------------------ | -------------------------------------- |
| [I/O][wasi-io] | Dan Gohman |
| [Filesystem][wasi-filesystem] | Dan Gohman |
| [Command-Line][wasi-command-line] | Dan Gohman |
| ["Classic" Command-Line][wasi-classic-command] | Dan Gohman |
| [Clocks][wasi-clocks] | Dan Gohman |
| [Random][wasi-random] | Dan Gohman |
| [Misc][wasi-misc] | Dan Gohman |
| [Handle Index][wasi-handle-index] | Dan Gohman |
| [Poll][wasi-poll] | Dan Gohman |
| [Machine Learning (wasi-nn)][wasi-nn] | Andrew Brown and Mingqiu Sun |

### Phase 1 - Feature Proposal (CG)
Expand All @@ -51,11 +52,13 @@ Proposals follow [this process document](https://github.com/WebAssembly/WASI/blo
Please see [Contributing to WebAssembly](https://github.com/WebAssembly/WASI/blob/master/Contributing.md) for the most up-to-date information on contributing proposals to standard.

[wasi-clocks]: https://github.com/WebAssembly/wasi-clocks
[wasi-command-line]: https://github.com/WebAssembly/wasi-classic-command
[wasi-classic-command]: https://github.com/WebAssembly/wasi-classic-command
[wasi-crypto]: https://github.com/WebAssembly/wasi-crypto
[wasi-filesystem]: https://github.com/WebAssembly/wasi-filesystem
[wasi-io]: https://github.com/WebAssembly/wasi-io
[wasi-misc]: https://github.com/WebAssembly/wasi-misc
[wasi-nn]: https://github.com/WebAssembly/wasi-nn
[wasi-proxy-wasm]: https://github.com/proxy-wasm/spec
[wasi-random]: https://github.com/WebAssembly/wasi-random
[wasi-handle-index]: https://github.com/WebAssembly/wasi-handle-index
[wasi-poll]: https://github.com/WebAssembly/wasi-poll
43 changes: 42 additions & 1 deletion meetings/2021/WASI-01-14.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,51 @@ Installation is required, see the calendar invite.
1. Please help add your name to the meeting notes.
1. Please help take notes.
1. Thanks!
1. Announcement
1. Pat will be changing branch name from `master` to `main`
1. Review of action items from previous meeting(s).
1. Dan report on repository organization progress
1. Lin update on repository organization progress
1. Proposals and discussions
1. Discussion: Testing guidelines for proposals
1. _Sumbit a PR to add your agenda item here_

## Meeting Notes
### Attendees
Lin Clark
Josh Dolitsky
Dan Gohman
Pat Hickey
Sam Clegg
Casper Beyer
Mark McCaskey
Johnnie Birch
Andrew Brown
Yong He
Peter Engelbert
Peter Huene
Martin Duke
Till Schneidereit
Bogus Wolf

For people taking notes, just type names; we’ll replace them with abbreviations before posting them publicly.

**Pat Hickey:** Github is now rolling out features to allow for renaming git branches from “master” to “main”, including blob url redirection. It’s mostly automatic, but it does require people to do some changes locally to update. We’ll put the steps in the WASI github issue tracking this.

Main topic: Testing.

**Lin Clark:** <slides>

The context here is WASI modularization -- breaking up the core WASI module into separate modules.

Pat and Dan are working on a new witx mechanism for expressing dependencies between witx files, allowing modules to depend on each other.

The main WASI repo will be the source of truth for stage-5 (standardized) proposals.

Tests should include both the source, in high-level languages, and compiled wasm binaries.

Things we still need to build:
A test runner
Tests. We’ll start with some tests from Wasmtime, and everyone can add tests.



177 changes: 174 additions & 3 deletions meetings/2021/WASI-01-28.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,182 @@ Installation is required, see the calendar invite.
1. Please help add your name to the meeting notes.
1. Please help take notes.
1. Thanks!
1. Review of action items from previous meeting(s).
1. Dan report on repository organization progress
1. Announcements
1. Lin Clark on WASI modularization milestone
1. Proposals and discussions
1. Presentation: Update from Luke Wagner on handles and resources in interface types
1. Discussion: Transitioning WASI APIs to use IT handles
1. _Sumbit a PR to add your agenda item here_

## Meeting Notes
### Attendees
- Martin Duke, F5
- Sam Clegg
- Dan Gohman
- Luke Wagner
- Mark McCaskey
- Lin Clark
- Dave Protasowski
- Alex Crichton
- Andrew Brown
- Mingqiu Sun
- Till Schneidereit
- Pat Hickey
- Josh Dolitsky
- Matt Butcher
- Ralph Squillace

### Announcements
**Lin Clark:** We now have a [milestone for WASI modularization](https://github.com/WebAssembly/WASI/milestone/1). Will be using milestones more in the future.

### Proposals and discussions
#### Presentation: Update from Luke Wagner on handles and resources in interface types [(Slides)](https://docs.google.com/presentation/d/1ikwS2Ps-KLXFofuS5VAs6Bn14q4LBEaxMjPfLj61UZE)

**Luke Wagner:** Handles and resources. Have been working on this for a while with a number of y’all
Give background, motivating problems, and proposed solution
Not at all final, still working on a PR. Your feedback is most welcome

IT gives WASI a bunch of value types
In addition to core basic types, a bunch of abbreviations
Don’t want to have to add core types
This lets us take a WASI function and use a bunch of high level types

Plan of record for a while has been abstract types for file descriptors
Type imports builds on reference types
Reference types are unforgeable, better than ints

How does this get virtualized?
Virtualization is a core value of WASI. You can implement with another WASI module
For this, we use module linking proposal

What does it look like to write that virtualized WASI? (walk through)
Packed the file index into the reference, that’s efficient

Type imports and exports are transparent by default
This is forgeable. That’s why that was take one

Type imports has a private type definition
I would take abstract type and make it a private type definition
Private.get and private.new only works inside module that defined the type definition
These private types have to be boxed.
Actually no worse than the native implementation to have one level of indirection

Two remaining problem areas: resource lifetimes and dynamic casting

Resource lifetimes
How do resources get released?
Could reference count.
Problematic for a variety of reasons.
Not virtualizable.
Because of downsides, WASI resources have explicit destructor

Virt WASI then looks like….
What happens on double close or use-after-close?
Spectrum of options, none of them good
How do I efficiently support cross-instance resource sharing?
Not great options either
How can a WASI impl robustly clean up after clients are destroyed?
Could leak.
Want FDs to be closed when the instance goes away, just like OS process
A native WASI impl could do this, but not a virtualized one
How do instances even get different lifetimes? This is a next step, to add ability to dynamically create and destroy instances
Also problems around dynamic casting
E.g. when store n distinct DOM node types in a single table
Not supporting would be a kind of regression

Dynamic casting breaks otherwise-static invariants
If there’s a run function and I pass a RO, is it possible for X to somehow do a write?
It would be nice to have that answer based only on the static type
Fix is to treat different imports as different things for purpose of dynamic cast
To fix this, you need a wrapping at some membrane/boundary
OK for core wasm, because that’s just a compiler target
But not ok for ecosystem

When we’re virtualizingWASI we want: …
Is all of this just for virtualization?
Resources are not a uniquely WASI concern
We should treat WASI as an unprivileged set of library interfaces

Proposal
Add a new interface type called handle
Handle is a value that can be copied
Refers to a resource
A resource is some entity with a lifetime. It can not be copied
So resource types are like abstract type, but only private option
Additionally resource type definitions can have an optional destructor
Resource creation and use is symmetric to private.new/get
Handles are semantically ref-counted and call the destructor when they get to 0
That’s what gives virt WASI same privileges as native
Also possible to explicitly destroy handles, and references trap

**Sam Clegg:** would need to be at WASI core?
**Luke Wagner:** So far everything in Interface Types

**Pat Hickey:** Attenuation
**Luke Wagner:** I’ve gone back and forth. Have an idea in open questions

**Luke Wagner:** High level summary of IT
They’re lazy copy operations
Need a way to box all interface types
Generally valuable because it solves interesting performance use cases
For handle values, creates a root that creates the stack
That makes it so handles can outlive stack frame

Once I have a reference, it can be cast
Will always trap if cast to differently privileged

IT gives us the boundary/membrane where to wrap

Proposal summary
Additions
New types
New definition
New instructions
Extend rtt.canon

Problems:
Robust and efficient handling of double-free/use-after-free
….

Open Questions
Add abstract subtyping?
Should we have specialized versions of handle (e.g. a strong handle, unique handle)

Next steps
Need to give lifetimes to instances
Can instances just be resources?
Could implement WASI command pattern in the toolchain rather than host magic

**Sam Clegg:** What happens if you destroy a resource and another module still has a copy

**Luke Wagner:** It will have a handle. That handle is then in a null state. When you try to use the handle, it would trap probably at resource.get

**Sam Clegg:** So the trap would happen in the callee

**Luke Wagner:** That’s where we might want to have a strong handle where no one could pass you an invalid handle. It’s an open question

**Ralph Squillace:** For what it’s worth, I’ve been spinning around Pat’s question and going back to DCE-RPC. Believe that you should make or signal a strong decision about shared resources.

**Luke Wagner:** That’s one of the reasons I like unique, because then when I give you a handle I’m explicitly saying I dont’ still have it. I guess that’s an argument for both unique and strong

**Ralph Squillace:** Having the ability to make those choices is what comes to mind. If I choose a convention, I can signal it to others. Previous technologies tried to let people do it without signaling capabilities

**Luke Wagner:** Would a summary be that you’re in favor of those two types of handles

**Ralph Squillace:** I would say those two handles express two slices of the problem space. I wouldn’t say that I’ve thought through other possible types.

**Luke Wagner:** Your’e right, I was trying to survey the literature and there are like 20 types

**Ralph Squillace:** Even if you don’t have strong type, at least you can trap. Maybe that’s the first step.

**Luke Wagner:** Hierarchy, undefined is worst, trap is better, statically preventing is best

**Sam Clegg:** These handles are at IT types and at core go to reference. So references can be copied

**Luke Wagner:** Once you exclusively use IT, there’s no back door

**Sam Clegg:** Any incremental steps?

**Luke Wagner:** In some sense, witx already has handle

**Lin Clark** proposes that we make that the main topic for next meeting

Loading