Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allowing LLVM to control data layout? #4

Open
wbrickner opened this issue Jun 7, 2022 · 0 comments
Open

Allowing LLVM to control data layout? #4

wbrickner opened this issue Jun 7, 2022 · 0 comments

Comments

@wbrickner
Copy link
Owner

Benefit

I've been thinking about how the input buffer and persistence layer has an arbitrary layout, and the exact way in which the components are arranged and accessed may determine LLVM's ability to optimize and especially its ability to vectorize cleanly.

I do not know how to write an algorithm that produces an optimal layout, or what that even is in general, but I suspect LLVM does.

The fundamental idea is to stop using buffers all together:

struct Net {
  persistence: [f32; 3]
}

becomes

struct Net {
  persistence_0: f32,
  persistence_1: f32,
  persistence_2: f32
}

the data no longer has any declared order, hopefully allowing LLVM to optimize the layout for us.
Of course, the size cannot be optimized, and it will always be optimal for the floats to be contiguous (just like an array), even if the struct exists as the daughter of another. The order is what would hopefully be optimizable.

Cost

  • This really fucks up the API for accessing recurrent state.
  • If applied to network input, unclear if beneficial, and supplying buffers is easier than providing a list of identifiers.
    • However, when querying a network of small input dimension, in practice, it is not uncommon for you to have the input components bound to local identifiers, which you then put into an array.
    • I would really enjoy it if we could optionally provide named components to a network, perhaps like:
.wind_speed(0.1)
.celsius(32.4)

Which is no longer demands representation as a flat buffer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant