Skip to content

Commit

Permalink
Improve precompiled downloads (#94)
Browse files Browse the repository at this point in the history
  • Loading branch information
jonatanklosko authored Aug 14, 2024
1 parent f9e2768 commit 06bdfd3
Show file tree
Hide file tree
Showing 8 changed files with 408 additions and 160 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,6 @@ xla-*.tar

# Output from compilations within Docker
/builds/output/

# Archive checksums file
/checksum.txt
6 changes: 0 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,6 @@ The target triplet describing the target platform, such as `aarch64-linux-gcc`.
this target is inferred for the host, however you may want to override this when cross-compiling
the project using Nerves.

#### `XLA_HTTP_HEADERS`

Headers to use when querying and downloading the precompiled archive. By default the
requests are sent to GitHub, unless `XLA_ARCHIVE_URL` specifies otherwise. The headers
should be a list following this format: `Key1: Value1; Key2: value2`.

## Building from source

> Note: currently only macOS and Linux is supported. When on Windows, the best option
Expand Down
41 changes: 41 additions & 0 deletions lib/mix/tasks/xla.checksum.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
defmodule Mix.Tasks.Xla.Checksum do
@moduledoc """
Generates a checksum file for all precompiled artifacts.
"""

use Mix.Task

@impl true
def run(_args) do
XLA.Utils.start_inets_profile()

Mix.shell().info("Downloading and computing checksums...")

checksums =
XLA.precompiled_files()
|> Task.async_stream(
fn {filename, url} ->
{filename, download_checksum!(url)}
end,
timeout: :infinity,
ordered: false
)
|> Map.new(fn {:ok, {filename, checksum}} -> {filename, checksum} end)

XLA.write_checksums!(checksums)

Mix.shell().info("Checksums written")
after
XLA.Utils.stop_inets_profile()
end

defp download_checksum!(url) do
case XLA.Utils.download(url, %XLA.Checksumer{}) do
{:ok, checksum} ->
checksum

{:error, message} ->
Mix.raise("failed to download archive from #{url}, reason: #{message}")
end
end
end
2 changes: 1 addition & 1 deletion lib/mix/tasks/xla.info.ex
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ defmodule Mix.Tasks.Xla.Info do
end

def run(["release_tag"]) do
Mix.shell().info(XLA.release_tag())
Mix.shell().info("v" <> XLA.version())
end

def run(["build_archive_dir"]) do
Expand Down
Loading

0 comments on commit 06bdfd3

Please sign in to comment.