Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation and add python docs #320

Merged
merged 1 commit into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 60 additions & 3 deletions docs/ARC.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ ARCtrl aims to provide an easy solution to create and manipulate ARCs in memory.

```fsharp
// F#
#r "nuget: FsSpreadsheet.ExcelIO, 5.0.2"
#r "nuget: ARCtrl, 1.0.0-beta.8"
#r "nuget: FsSpreadsheet.Net"
#r "nuget: ARCtrl"

open ARCtrl

Expand All @@ -31,6 +31,13 @@ import {ARC} from "@nfdi4plants/arctrl";
let arc = new ARC()
```

```python
# Python
from arctrl.arc import ARC

myArc = ARC()
```

This will initialize an ARC without metadata but with the basic ARC folder structure in `arc.FileSystem`

- 📁 ARC root
Expand All @@ -48,7 +55,7 @@ In .NET you can use [ARCtrl.NET][1] to handle any contract based read/write oper
// F#
open ARCtrl.Contract
open FsSpreadsheet
open FsSpreadsheet.ExcelIO
open FsSpreadsheet.Net

let arcRootPath = @"path/where/you/want/the/NewTestARC"

Expand Down Expand Up @@ -76,6 +83,19 @@ async function write(arcPath, arc) {
await write(arcRootPath, arc)
```

```python
# Python
from arctrl.arc import ARC
from Contract import fulfill_write_contract, fulfill_read_contract

async def write(arc_path, arc):
contracts = arc.GetWriteContracts()
for contract in contracts:
# from Contracts.js docs
await fulfill_write_contract(arc_path, contract)
```


## Read

Read may look intimidating at first, until you notice that most of this is just setup which can be reused for any read you do.
Expand Down Expand Up @@ -174,5 +194,42 @@ async function read(basePath) {

await read(arcRootPath).then(arc => console.log(arc))
```
```python
import os
from arctrl.arc import ARC
from Contract import fulfill_write_contract, fulfill_read_contract

#Python
def normalize_path_separators(path_str):
normalized_path = os.path.normpath(path_str)
return normalized_path.replace('\\', '/')

def get_all_file_paths(base_path):
files_list = []
def loop(dir_path):
files = os.listdir(dir_path)
for file_name in files:
file_path = os.path.join(dir_path, file_name)
if os.path.isdir(file_path):
loop(file_path)
else:
relative_path = os.path.relpath(file_path, base_path)
normalize_path = normalize_path_separators(relative_path)
files_list.append(normalize_path)
loop(base_path)
return files_list

# put it all together
def read(base_path):
all_file_paths = get_all_file_paths(base_path)
arc = ARC.from_file_paths(all_file_paths)
read_contracts = arc.GetReadContracts()
print(read_contracts)
fcontracts = (fulfill_read_contract(base_path, contract) for contract in read_contracts)
for contract, content in zip(read_contracts, fcontracts):
contract.DTO = content
arc.SetISAFromContracts(fcontracts)
return arc

```
[1]: <https://www.nuget.org/packages/ARCtrl.NET> "ARCtrl.NET Nuget"
50 changes: 47 additions & 3 deletions docs/ArcInvestigation.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ Comments can be used to add freetext information to the Investigation metadata s

```fsharp
// F#
#r "nuget: FsSpreadsheet.ExcelIO, 5.0.2"
#r "nuget: ARCtrl, 1.0.0-beta.8"
#r "nuget: FsSpreadsheet.Net"
#r "nuget: ARCtrl"

open ARCtrl.ISA

Expand All @@ -76,7 +76,21 @@ investigation_comments.Comments.push(newComment2)

console.log(investigation_comments)
```
```python
#Python
from arctrl.arctrl import ArcInvestigation, Comment

# Comments
investigation_comments = ArcInvestigation.init("My Investigation")

new_comment = Comment.create("The Id", "The Name", "The Value")
new_comment2 = Comment.create("My other ID", "My other Name", "My other Value")

investigation_comments.Comments.append(new_comment)
investigation_comments.Comments.append(new_comment2)

print(investigation_comments)
```
This code example will produce the following output after writing to `.xlsx`.

| INVESTIGATION | |
Expand All @@ -101,7 +115,7 @@ This code example will produce the following output after writing to `.xlsx`.
```fsharp
// F#
open ARCtrl.ISA.Spreadsheet
open FsSpreadsheet.ExcelIO
open FsSpreadsheet.Net

let fswb = ArcInvestigation.toFsWorkbook investigation_comments

Expand All @@ -117,6 +131,16 @@ let fswb = toFsWorkbook(investigation_comments)

Xlsx.toFile("test.isa.investigation.xlsx", fswb)
```
```python
# Python
from fsspreadsheet.xlsx import Xlsx
from arctrl.ISA.ISA_Spreadsheet.arc_investigation import to_fs_workbook, from_fs_workbook

fswb = to_fs_workbook(investigation_comments)

Xlsx.to_file("test.isa.investigation.xlsx", fswb)
```


## Json

Expand Down Expand Up @@ -147,6 +171,16 @@ const json = ArcInvestigation_toJsonString(investigation)

console.log(json)
```
```python
# Python
from arctrl.ISA.ISA_Json.ArcTypes.arc_investigation import ArcInvestigation_toJsonString, ArcInvestigation_fromJsonString

investigation = ArcInvestigation.init("My Investigation")

json_str = ArcInvestigation_toJsonString(investigation)

print(json_str)
```

### Read Json

Expand Down Expand Up @@ -174,4 +208,14 @@ const jsonString = json
const investigation_2 = ArcInvestigation_fromJsonString(jsonString)

console.log(investigation_2)
```
```python
# Python
from arctrl.ISA.ISA_Json.ArcTypes.arc_investigation import ArcInvestigation_toJsonString, ArcInvestigation_fromJsonString

json_string = json_str

investigation_2 = ArcInvestigation_fromJsonString(json_string)

print(investigation_2.Equals(investigation))
```
54 changes: 50 additions & 4 deletions docs/Contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,19 @@ export class Contract extends Record {
this.DTO = DTO;
}
```
```python
# Python
class Contract(Record):
# Can be any of: "CREATE", "UPDATE", "DELETE", "READ", "EXECUTE"
Operation: str
# string, the path where the io operation should be executed. The path is relative to ARC root
Path: str
# Can be undefined or any of: "ISA_Assay", "ISA_Study", "ISA_Investigation", "JSON", "Markdown", "CWL", "PlainText", "Cli".
DTOType: DTOType | None
# Can be undefined or any of: `FsWorkbook` (from fsspreadsheet), string (e.g. json,..), or a `CLITool`.
DTO: DTO | None
```


Handling contracts can be generalized in a few functions.

Expand All @@ -57,15 +70,17 @@ Next we need to know how to handle our DTO. In .NET this is implemented as Discr
In both languages we must specify how spreadsheet objects and plain text objects are correctly handled.

```fsharp
#r "nuget: FsSpreadsheet.ExcelIO, 5.0.2"
#r "nuget: ARCtrl, 1.0.0-beta.8"
// FSharp
#r "nuget: FsSpreadsheet.Net"
#r "nuget: ARCtrl"

open ARCtrl
open ARCtrl.Contract
open FsSpreadsheet
open FsSpreadsheet.ExcelIO
open FsSpreadsheet.Net

/// From ARCtrl.NET
/// https://github.com/nfdi4plants/ARCtrl.NET/blob/f3eda8e96a3a7791288c1b5975050742c1d803d9/src/ARCtrl.NET/Contract.fs#L24
let fulfillWriteContract basePath (c : Contract) =
let ensureDirectory (filePath : string) =
let file = new System.IO.FileInfo(filePath);
Expand All @@ -74,7 +89,7 @@ let fulfillWriteContract basePath (c : Contract) =
| Some (DTO.Spreadsheet wb) ->
let path = System.IO.Path.Combine(basePath, c.Path)
ensureDirectory path
FsWorkbook.toFile path (wb :?> FsWorkbook)
FsWorkbook.toXlsxFile path (wb :?> FsWorkbook)
| Some (DTO.Text t) ->
let path = System.IO.Path.Combine(basePath, c.Path)
ensureDirectory path
Expand Down Expand Up @@ -117,6 +132,37 @@ export async function fulfillWriteContract (basePath, contract) {
}
}
```
```python
import {Xlsx} from "fsspreadsheet";
import fs from "fs";
import path from "path";

export async function fulfillWriteContract (basePath, contract) {
function ensureDirectory (filePath) {
let dirPath = path.dirname(filePath)
if (!fs.existsSync(dirPath)){
fs.mkdirSync(dirPath, { recursive: true });
}
}
const p = path.join(basePath,contract.Path)
if (contract.Operation = "CREATE") {
if (contract.DTO == undefined) {
ensureDirectory(p)
fs.writeFileSync(p, "")
} else if (contract.DTOType == "ISA_Assay" || contract.DTOType == "ISA_Study" || contract.DTOType == "ISA_Investigation") {
ensureDirectory(p)
await Xlsx.toFile(p, contract.DTO)
console.log("ISA", p)
} else if (contract.DTOType == "PlainText") {
ensureDirectory(p)
fs.writeFileSync(p, contract.DTO)
} else {
console.log("Warning: The given contract is not a correct ARC write contract: ", contract)
}
}
}
```



## READ contracts
Expand Down
4 changes: 2 additions & 2 deletions docs/DesignPrinciples.md
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,7 @@ end

subgraph IO
node[node.fs]
fsspreadx[FsSpreadsheet.ExcelIO]
fsspreadx[FsSpreadsheet.Net]
excelJS[exceljs]
systemio[System.IO]
end
Expand Down Expand Up @@ -539,7 +539,7 @@ class cwl["CWL"] {
Workflows
}
class io["IO"] {
FsSpreadsheet.ExcelIO
FsSpreadsheet.Net
System.IO
exceljs
node fs
Expand Down
36 changes: 31 additions & 5 deletions docs/GettingStarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@

1. Install [.NET SDK](https://dotnet.microsoft.com/en-us/download).
2. Get the [ARCtrl nuget package](www.nuget.org/packages/ARCtrl).
3. (*OPTIONAL*) Use [FsSpreadsheet.ExcelIO](https://www.nuget.org/packages/FsSpreadsheet.ExcelIO) to get any xlsx files into the correct format with our supported readers.
3. (*OPTIONAL*) Use [FsSpreadsheet.Net](https://www.nuget.org/packages/FsSpreadsheet.Net) to get any xlsx files into the correct format with our supported readers.
4. (*OPTIONAL*) You can use contract handling from [ARCtrl.NET](https://www.nuget.org/packages/ARCtrl.NET). This documentation will avoid using ARCtrl.NET and extract the relevant functions.

Thats it! 🎉

For any documentation we assume using ARCtrl from a .fsx file. Verify correct setup by creating a `ARC.fsx` file with the following content

```fsharp
#r "nuget: FsSpreadsheet.ExcelIO, 5.0.2"
#r "nuget: ARCtrl, 1.0.0-beta.8"
#r "nuget: FsSpreadsheet.Net"
#r "nuget: ARCtrl"

open ARCtrl

Expand Down Expand Up @@ -54,8 +54,8 @@ Your `package.json` might look similiar to this:
"author": "Kevin Frey",
"license": "MIT",
"dependencies": {
"@nfdi4plants/arctrl": "^1.0.0-alpha9",
"fsspreadsheet": "^4.0.0-alpha2"
"@nfdi4plants/arctrl": "^1.2.0",
"fsspreadsheet": "^5.2.0"
}
}
```
Expand All @@ -69,4 +69,30 @@ Verify correct setup by creating `ARC.js` file with the content from below in th
import {ARC} from "@nfdi4plants/arctrl";

console.log(ARC) // [class ARC]
```

## Setup - Python

1. Install [python >3.11.x](https://www.python.org/downloads/)
2. Create folder and put an `requirements.txt` file inside.
3. Write `arctrl` and `fsspreadsheet` in two separate lines into it
```
arctrl
fsspreadsheet
```
4. run `py -m pip install -r requirements.txt`

Of course you can replace the command `py` with anything that leads to the python executable of your liking.

Thats it! 🎉

You can now reference ARCtrl in any `.py` file and run it with `py path/to/Any.py`.

Verify correct setup by creating `ARCTest.py` file with the content from below in the same folder, which contains your `requirements.txt`. Then run `py ./ArcTest.py`. This will print `<class 'arctrl.arc.ARC'>` into the console.

```python
// ARCTest.py
from arctrl.arctrl import ARC;

print(ARC) // <class 'arctrl.arc.ARC'>
```
6 changes: 3 additions & 3 deletions docs/scripts_fsharp/ARC.fsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#r "nuget: FsSpreadsheet.ExcelIO, 5.0.2"
#r "nuget: ARCtrl, 1.0.0-beta.8"
#r "nuget: FsSpreadsheet.Net"
#r "nuget: ARCtrl"
#load "Contracts.fsx"

// # Create
Expand All @@ -15,7 +15,7 @@ arc.FileSystem

open ARCtrl.Contract
open FsSpreadsheet
open FsSpreadsheet.ExcelIO
open FsSpreadsheet.Net

let arcRootPath = @"C:\Users\Kevin\Desktop\NewTestARCNET"

Expand Down
6 changes: 3 additions & 3 deletions docs/scripts_fsharp/ArcInvestigation.fsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#r "nuget: FsSpreadsheet.ExcelIO, 5.0.2"
#r "nuget: ARCtrl, 1.0.0-beta.9"
#r "nuget: FsSpreadsheet.Net"
#r "nuget: ARCtrl"

open ARCtrl.ISA

Expand All @@ -16,7 +16,7 @@ investigation_comments.Comments <- Array.append investigation_comments.Comments

// ## Xlsx - Write
open ARCtrl.ISA.Spreadsheet
open FsSpreadsheet.ExcelIO
open FsSpreadsheet.Net

let fswb = ArcInvestigation.toFsWorkbook investigation_comments

Expand Down
Loading
Loading