Skip to content

Commit

Permalink
Develop of Theta Data Provider (#1)
Browse files Browse the repository at this point in the history
* update: .gitignore

* update: ignore `Data/*` folder in .gitignore

* feat: prepare structure of data folders

* feat: workflow and GH's templates

* feat: DataQueueHandler for Option

* feat: prepare Test project

* feat: test for DQH

* feat: symbolMapper

* feat: Contract WS model

* feat: Handle Trade/Quote response with aggregator

* feat: convert string to Date extension

* feat: IsConnected flag from ws connection

* feat: ITimeProvider in DataProvider

* feat: ThetaData incapsulate Rest API Client logic

* feat: IDataQueueUniverseProvider && IOptionChainProvider
feat: GetLean in SymbolMapper
feat: generic entity for REST responses
feat: test for IDataQueueUniverseProvider and IOptionChainProvider

* remove: not used usings

* refactor: SymbolMapper + tests

* rename: extension converting date

* rename: RestResponse to BaseResponse

* refactor: RestApi BaseUrl

* feat: free space command run in workflow file

* move: BaseResponse to Common folder

* feat: log of requested url

* refactor: global variable of restClient and add comment

* feat: first impl GetHistory Option

* refactor: symbolMapper + new tests

* refactor: symbolMapper use Market USA

* feat: create symbol extension in test

* feat: DQH additional test cases

* feat: exception msg in converter classes

* refactor: short type to byte in EOD entity

* fix: validation of OpenInterest tickType

* refactor: GetHistory of Daily TickType data
feat: GetHistory Tests

* feat: validation of correct status code response

* feat: add validation subscription on IndexOption symbol

* fix: missed double quotes

* feat: support of muliple pages requests/responses
feat: base entity for Header of Responses
feat: json null string converter cuz API return "null" like a a string

* feat: Exchanges' code collection
remove: condition collection
refactor: some property in entities

* feat: exchange's code to WS responses

* fix: convert time in history request

* feat: GetHistory tests

* feat: DataDownloader
refactor: history test
refactor: DataQueueUniverse provider

* refactor: skip empty Response for Trade of Quote Daily

* feat: xml description RestApiClient

* feat: Trade(Tick,Min,Hour) DataConsolidator
refactor: test's GetHistory

* refactor: IOptionChainProvider and IDataQueueUniverseProvider

* feat: add json description file

* refactor: use Lean consolidator for trade ticks

* fix: OptionStyle in GetOptionContractList

* refactor: LookupSymbols
remove: GetTickTime()
remove: not use variables

* remove: CachingOptionChainProvider for OptionChainProvider instance

* revert: not use param to explicit in LookupSymbols

* remove: duplicate validation in OptionChain
feat: test option future chains

* feat: ValidateSubscription()

* feat: additional test case with invalid param of GetHIstory

* feat: implement different subscription price plan

* fix: wrong validation of available subscription process

* refactor:test: DQH multiple subscription

* refactor: use custom convert for Date From API

* refactor: use generic format for date ThetaData extension

* feat: Custom Json convert For ThetaData DateTime format from WS
  • Loading branch information
Romazes authored Apr 5, 2024
1 parent 7c455c4 commit bfb96fa
Show file tree
Hide file tree
Showing 52 changed files with 4,547 additions and 21 deletions.
27 changes: 27 additions & 0 deletions .github/issue_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<!--- This template provides sections for bugs and features. Please delete any irrelevant sections before submitting -->

#### Expected Behavior
<!--- Required. Describe the behavior you expect to see for your case. -->

#### Actual Behavior
<!--- Required. Describe the actual behavior for your case. -->

#### Potential Solution
<!--- Optional. Describe any potential solutions and/or thoughts as to what may be causing the difference between expected and actual behavior. -->

#### Reproducing the Problem
<!--- Required for Bugs. Describe how to reproduce the problem. This can be via a failing unit test or a simplified algorithm that reliably demonstrates this issue. -->

#### System Information
<!--- Required for Bugs. Include any system specific information, such as OS. -->

#### Checklist
<!--- Confirm that you've provided all the required information. -->
<!--- Required fields --->
- [ ] I have completely filled out this template
- [ ] I have confirmed that this issue exists on the current `master` branch
- [ ] I have confirmed that this is not a duplicate issue by searching [issues](https://github.com/QuantConnect/Lean/issues)
<!--- Required for Bugs, feature request can delete the line below. -->
- [ ] I have provided detailed steps to reproduce the issue

<!--- Template inspired by https://github.com/stevemao/github-issue-templates -->
42 changes: 42 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@

<!--- Provide a general summary of your changes in the Title above -->

#### Description
<!--- Describe your changes in detail -->

#### Related Issue
<!--- This project only accepts pull requests related to open issues -->
<!--- If suggesting a new feature or change, please discuss it in an issue first -->
<!--- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!--- Please link to the issue here: -->

#### Motivation and Context
<!--- Why is this change required? What problem does it solve? -->

#### Requires Documentation Change
<!--- Please indicate if these changes will require updates to documentation, and if so, specify what changes are required -->

#### How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->

#### Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Refactor (non-breaking change which improves implementation)
- [ ] Performance (non-breaking change which improves performance. Please add associated performance test and results)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Non-functional change (xml comments/documentation/etc)

#### Checklist:
<!--- The following is a checklist of items that MUST be completed before a PR is accepted -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] I have read the **CONTRIBUTING** [document](https://github.com/QuantConnect/Lean/blob/master/CONTRIBUTING.md).
- [ ] I have added tests to cover my changes. <!--- If not applicable, please explain why -->
- [ ] All new and existing tests passed.
- [ ] My branch follows the naming convention `bug-<issue#>-<description>` or `feature-<issue#>-<description>`

<!--- Template inspired by https://www.talater.com/open-source-templates/#/page/99 -->
49 changes: 49 additions & 0 deletions .github/workflows/gh-actions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: Build & Test

on:
push:
branches: ["*"]

jobs:
build:
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Free space
run: df -h && sudo rm -rf /usr/local/lib/android && sudo rm -rf /opt/ghc && rm -rf /opt/hostedtoolcache* && df -h

- name: Checkout Lean Same Branch
id: lean-same-branch
uses: actions/checkout@v2
continue-on-error: true
with:
ref: ${{ github.ref }}
repository: QuantConnect/Lean
path: Lean

- name: Checkout Lean Master
if: steps.lean-same-branch.outcome != 'success'
uses: actions/checkout@v2
with:
repository: QuantConnect/Lean
path: Lean

- name: Move Lean
run: mv Lean ../Lean

- name: Run Image
uses: addnab/docker-run-action@v3
with:
image: quantconnect/lean:foundation
options: -v /home/runner/work:/__w --workdir /__w/Lean.DataSource.ThetaData/Lean.DataSource.ThetaData -e QC_THETADATA_USERNAME=${{ secrets.THETADATA_USERNAME }} -e QC_THETADATA_PASSWORD=${{ secrets.THETADATA_PASSWORD }} -e QC_JOB_USER_ID=${{ secrets.JOB_USER_ID }} -e QC_API_ACCESS_TOKEN=${{ secrets.API_ACCESS_TOKEN }} -e QC_JOB_ORGANIZATION_ID=${{ secrets.JOB_ORGANIZATION_ID }}

- name: Build QuantConnect.ThetaData
run: dotnet build ./QuantConnect.ThetaData/QuantConnect.DataSource.ThetaData.csproj /p:Configuration=Release /v:quiet /p:WarningLevel=1

- name: Build QuantConnect.ThetaData.Tests
run: dotnet build ./QuantConnect.ThetaData.Tests/QuantConnect.DataSource.ThetaData.Tests.csproj /p:Configuration=Release /v:quiet /p:WarningLevel=1

- name: Run QuantConnect.ThetaData.Tests
run: dotnet test ./QuantConnect.ThetaData.Tests/bin/Release/QuantConnect.Lean.DataSource.ThetaData.dll
Loading

0 comments on commit bfb96fa

Please sign in to comment.