Skip to content

Commit

Permalink
Acknowledgement in NOTICE file (#287)
Browse files Browse the repository at this point in the history
## Summary
Acknowledgement in NOTICE file

<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->

## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->

<!-- 
Replace BLANK with your device type. For example, A100-80G-PCIe

Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them. 
-->

- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence

---------

Co-authored-by: Shao Tang <tangshao28@gmail.com>
  • Loading branch information
momochen and lancerts authored Oct 1, 2024
1 parent 1520999 commit d4933b5
Showing 1 changed file with 54 additions and 0 deletions.
54 changes: 54 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,57 @@ Copyright 2024 LinkedIn Corporation
All Rights Reserved.

Licensed under the BSD 2-Clause License (the "License"). See License in the project root for license information.

This product includes software developed by LinkedIn Corporation.

This product contains code derived from the following open source projects:

1. Unsloth
Copyright (c) 2023 Unsloth AI
Licensed under the Apache License, Version 2.0
Source: https://github.com/unslothai/unsloth

The `calculate_settings` function to determine block size and warp is reused for Norm and MLP operations.
Modifications and additions were made to the RMS Norm implementation.

2. Triton
Copyright (c) 2023 OpenAI
Licensed under the MIT License
Source: https://github.com/openai/triton

Modifications were made based on Triton tutorials for the RMS Norm implementation.

3. Efficient Cross Entropy
Copyright (c) 2023 Mohamed Malek
Licensed under the MIT License
Source: https://github.com/mgmalek/efficient_cross_entropy

The idea of gradient-in-forward and chunking was used in the Linear Cross Entropy implementation.

4. Flash Attention
Copyright (c) 2023 Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré
Licensed under the BSD 3-Clause License
Source: https://github.com/Dao-AILab/flash-attention

Optimization ideas such as tiling and recomputation were inspired by this work.

5. AutoAWQ
Copyright (c) 2023 Casper Hansen
Licensed under the MIT License
Source: https://github.com/casper-hansen/AutoAWQ

The design of the automodel was referenced from this project.

6. llm.c
Copyright (c) 2023 Andrej Karpathy
Licensed under the MIT License
Source: https://github.com/karpathy/llm.c

The design of end-to-end testing was referenced from this project.

7. Tiny Shakespeare Dataset
Source: https://huggingface.co/datasets/karpathy/tiny_shakespeare

This dataset is used to conduct convergence tests on mini models.

For full license texts, please refer to the respective project repositories.

0 comments on commit d4933b5

Please sign in to comment.