Skip to content

Commit

Permalink
Add more fill() constructor functions.
Browse files Browse the repository at this point in the history
Also:
-Improve coverage.
-Improve docs.
-Add ones() and zeros() constructor functions.
-Bump to v0.3.4.
  • Loading branch information
ma-laforge committed Mar 4, 2021
1 parent bba89a7 commit 449b451
Show file tree
Hide file tree
Showing 10 changed files with 275 additions and 63 deletions.
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name = "MDDatasets"
uuid = "176e4ad4-9107-502a-977c-53a5728fe704"
version = "0.3.3"
version = "0.3.4"

[deps]
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
Expand Down
29 changes: 16 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
1. [demo1](doc/demo1.md)
1. [Core architecture](doc/architecture.md)
1. [Principal types](doc/architecture.md#PrincipalTypes)
1. [Object construction](doc/architecture.md#Constructors)
1. [Functions of 1 argument (`DataF1`) & interpolation](doc/architecture.md#F1Arg)
1. [Multi-dimensional datasets (`DataRS`) & broadcasting](doc/architecture.md#MDDatasets)
1. [Supported functions](doc/architecture.md#SupportedFunctions)
Expand Down Expand Up @@ -54,24 +55,26 @@ More advanced usage examples can be found in the sample directories of [`CMDimDa
## Usage: Constructing a recursive-sweep dataset

Assuming input data can be generated using the following:
```julia
t = DataF1((0:.01:10)*1e-9) #Time vector stored as a function of 1 argument

t = DataF1((0:.01:10)*1e-9) #Time vector stored as a function of 1 argument

#NOTE: get_ydata returns type "DataF1" (stores data as a function of 1 argument):
get_ydata(t::DataF1, tbit, vdd, trise) = sin(2pi*t/tbit)*(trise/tbit)+vdd
#NOTE: get_ydata returns type "DataF1" (stores data as a function of 1 argument):
get_ydata(t::DataF1, tbit, vdd, trise) = sin(2pi*t/tbit)*(trise/tbit)+vdd
```

One can create a relatively complex Recursive-Sweep (DataRS) dataset using the following pattern:

datars = fill(DataRS, PSweep("tbit", [1, 3, 9] * 1e-9)) do tbit
fill(DataRS, PSweep("VDD", 0.9 * [0.9, 1, 1.1])) do vdd

#Inner-most sweep: need to specify element type (DataF1):
#(Other (scalar) element types: DataInt/DataFloat/DataComplex)
fill(DataRS{DataF1}, PSweep("trise", [0.1, 0.15, 0.2] * tbit)) do trise
return get_ydata(t, tbit, vdd, trise)
end
```julia
datars = fill(DataRS, PSweep("tbit", [1, 3, 9] * 1e-9)) do tbit
fill(DataRS, PSweep("VDD", 0.9 * [0.9, 1, 1.1])) do vdd

#Inner-most sweep: need to specify element type (DataF1):
#(Other (scalar) element types: DataInt/DataFloat/DataComplex)
fill(DataRS{DataF1}, PSweep("trise", [0.1, 0.15, 0.2] * tbit)) do trise
return get_ydata(t, tbit, vdd, trise)
end
end
end
```

<a name="KnownLimitations"></a>
## Known limitations
Expand Down
87 changes: 84 additions & 3 deletions doc/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,90 @@
- **`DataRS{DataF1/DataInt/DataFloat/DataComplex}`**: A recursive-sweep organization of data. Principally designed to collect massive datasets with *dependent* control variables([see examples](#SampleUsage_DataRS)).
- **`PSweep`**: A parameter sweep (i.e. an independent control variable that generates experimental points in a `DataRS/DataHR` dataset).

<a name="Constructors"></a>
## Object construction
Basic definitions are provided in this section. See [Sample usage](../README.md#SampleUsage) section for more examples.

### Construction: `DataF1`
- `DataF1(x::Vector, y::Vector)`: Basic constructor
- `DataF1([1,2,3], [1,4,9])`
- `DataF1(rng::AbstractRange)`: Both x & y values are set to `collect(1:10)`
- `DataF1(1:10)`: `x = y = collect(1:10)`

### Construction: `PSweep`
Individual `PSweep` (parameter sweeps) are of single-dimension **only**:
- `PSweep(id::String, sweep::Vector)`: Basic constructor
- `PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)`

Mult-parameter sweeps are defined as a `PSweep[]` (`Vector`):
```julia
sweeplist = PSweep[
PSweep("A", [1, 2, 4])
PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)
PSweep("phi", π .* [1/4, 1/3, 1/2])
]
```

### Construction: `DataRS`
Reminder: DataRS is a Recursive-Sweep data container.

Basic construction:
- `fill(val::Number/DataF1, DataRS, sweeplist::Vector{PSweep})`
- `fill(32.0, DataRS, sweeplist)`
- `fill(DataF1(1:10), DataRS, sweeplist)`

Generating a relatively complex, initial dataset is easy:
```julia
Tsim = 2e-3 #2ms

sweeplist = PSweep[
PSweep("A", [1, 2, 4])
PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)
PSweep("phi", π .* [1/4, 1/3, 1/2])
]

#Get t values for all swept parameters (DataRS structure):
t = fill(DataF1(range(0,Tsim,length=1000)), DataRS, sweeplist)

#Extract values of each parameter:
𝑓 = parameter(t, "freq")
A = parameter(t, "A")
𝜙 = parameter(t, "phi")

#Compute sinusoidal signal for each combination of swept parameter:
𝜔 = 2π*𝑓 #Convenience
sig = A * sin(𝜔*t + 𝜙)

```

ydata = fill(DataRS{DataF1}, PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)) do 𝑓
𝜔 = 2π*𝑓; T = 1/𝑓; 𝜙 = 0; A = 1.2
sig = A * sin(𝜔*t + 𝜙)
return sig
end;

#### Construction: `DataRS` (read in data from simulation)
A more flexible way to construct a `DataRS` structure is with the following pattern:
```julia
#Emulate results of a simulation:
get_ydata(t, A, 𝑓, 𝜙) = A * sin((2π*𝑓)*t + 𝜙)

Tsim = 2e-3 #2ms
t = DataF1(range(0,Tsim,length=1000))

sig = fill(DataRS, PSweep("A", [1, 2, 4])) do A
fill(DataRS, PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)) do 𝑓

#Inner-most sweep: need to specify element type (DataF1):
#(or other (scalar) element types, when desired: DataInt/DataFloat/DataComplex)
fill(DataRS{DataF1}, PSweep("phi", π .* [1/4, 1/3, 1/2])) do 𝜙
return get_ydata(t, A, 𝑓, 𝜙)
end
end
end
```
This construction method is particularly well suited to read in data from a simulation.

<a name="F1Arg"></a>
## Functions of 1 argument (`DataF1`) & interpolation

Expand Down Expand Up @@ -54,9 +138,6 @@ TODO: Provide a means to re-order dimensions.
<a name="SupportedFunctions"></a>
# Supported functions

## Constructors
- **`fill`**`()`: Create a DataHR or DataRS structure (see examples for use).

## Helper functions

- **`ensure`**`(cond, err)`: Similar to assert, but will never compile out (not just for debugging).
Expand Down
1 change: 1 addition & 0 deletions src/MDDatasets.jl
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ include("broadcast_datars.jl")
include("datasetop_reg.jl")
include("measinterface.jl")
include("show.jl")
include("doc.jl")

#==TODO: Watch out for value() being exported by multiple modules...
Maybe it can be defined in "Units"
Expand Down
24 changes: 19 additions & 5 deletions src/datahr.jl
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ mutable struct DataHR{T} <: DataMD
if !elemallowed(DataMD, eltype(elem))
msg = "Can only create DataHR{T} for T ∈ {DataF1, DataFloat, DataInt, DataComplex}"
throw(ArgumentError(msg))
elseif ndims(DataHR, sweeps) != ndims(elem)
elseif size(DataHR, sweeps) != size(elem)
throw(ArgumentError("Number of sweeps must match dimensionality of elem"))
end
return new(sweeps, elem)
Expand Down Expand Up @@ -133,10 +133,24 @@ function Base.fill!(fn::Function, d::DataHR)
end
return d
end
Base.fill(fn::Function, ::Type{DataHR{T}}, sweeps::Vector{PSweep}) where T =
fill!(fn, DataHR{T}(sweeps))
Base.fill(fn::Function, ::Type{DataHR}, sweeps::Vector{PSweep}) = fill(fn, DataHR{DataF1}, sweeps)
Base.fill(fn::Function, ::Type{DataHR{T}}, sweep::PSweep) where T = fill(fn, DataHR{DataF1}, PSweep[sweep])
Base.fill(fn::Function, ::Type{DataHR{T}}, swlist::Vector{PSweep}) where T =
fill!(fn, DataHR{T}(swlist))
Base.fill(fn::Function, ::Type{DataHR{T}}, sweep::PSweep) where T = fill(fn, DataHR{T}, PSweep[sweep])
#Default: DataHR -> creates DataHR{DataF1}??? Is this a bad idea?
Base.fill(fn::Function, ::Type{DataHR}, swlist::Vector{PSweep}) = fill(fn, DataHR{DataF1}, swlist)

#Fill with a specified value:
Base.fill(elem_0::Number, ::Type{DataHR}, swlist::Vector{PSweep}) =
DataHR(swlist, fill(elem_0, size(DataHR, swlist)))
Base.fill(elem_0::DataF1, ::Type{DataHR}, swlist::Vector{PSweep}) =
DataHR(swlist, fill!(Array{DataF1}(undef, size(DataHR, swlist)), elem_0)) #Must be array of abstract DatF1[]
Base.fill(elem_0::DF1_Num, ::Type{DataHR}, sweep::PSweep) = fill(elem_0, DataHR, PSweep[sweep])

#zeros & ones:
Base.zeros(::Type{DataHR}, swlist::Vector{PSweep}) = fill(0.0, DataHR, swlist)
Base.zeros(::Type{DataHR}, sweep::PSweep) = zeros(DataHR, PSweep[sweep])
Base.ones(::Type{DataHR}, swlist::Vector{PSweep}) = fill(1.0, DataHR, swlist)
Base.ones(::Type{DataHR}, sweep::PSweep) = ones(DataHR, PSweep[sweep])


#==Data generation
Expand Down
35 changes: 12 additions & 23 deletions src/datars.jl
Original file line number Diff line number Diff line change
Expand Up @@ -190,29 +190,6 @@ end
#==Help with construction
===============================================================================#

@doc """
fill(d::DataRS, ...)
Construct a DataRS structure storing results from parametric sweeps using recursive data structures.
# Examples
```julia-repl
signal = fill(DataRS, PSweep("A", [1, 2, 4] .* 1e-3)) do A
fill(DataRS, PSweep("phi", [0, 0.5, 1] .* (π/4))) do 𝜙
fill(DataRS{DataF1}, PSweep("freq", [1, 4, 16] .* 1e3)) do 𝑓
𝜔 = 2π*𝑓; T = 1/𝑓
Δt = T/100 #Define resolution from # of samples per period
Tsim = 4T #Simulated time
t = DataF1(0:Δt:Tsim) #DataF1 creates a t:{y, x} container with y == x
sig = A * sin(𝜔*t + 𝜙) #Still a DataF1 sig:{y, x=t} container
return sig
end; end; end
```
Note that inner-most sweep needs to specify element type (DataF1).
Other (scalar) element types include: DataInt/DataFloat/DataComplex.
""" Base.fill(::DataRS, args...)

#Implement "fill(DataRS, ...) do sweepval" syntax:
function Base.fill!(fn::Function, d::DataRS)
for i in 1:length(d.sweep)
Expand All @@ -225,6 +202,18 @@ Base.fill(fn::Function, ::Type{DataRS{T}}, sweep::PSweep) where T =
Base.fill(fn::Function, ::Type{DataRS}, sweep::PSweep) = fill(fn, DataRS{DataRS}, sweep)


#Fill with a specified value (leverage DataHR algorithms for now):
Base.fill(elem_0::DF1_Num, ::Type{DataRS}, swlist::Vector{PSweep}) =
convert(DataRS, fill(elem_0, DataHR, swlist))
Base.fill(elem_0::DF1_Num, ::Type{DataRS}, sweep::PSweep) = fill(elem_0, DataRS, PSweep[sweep])

#zeros & ones:
Base.zeros(::Type{DataRS}, swlist::Vector{PSweep}) = fill(0.0, DataRS, swlist)
Base.zeros(::Type{DataRS}, sweep::PSweep) = zeros(DataRS, PSweep[sweep])
Base.ones(::Type{DataRS}, swlist::Vector{PSweep}) = fill(1.0, DataRS, swlist)
Base.ones(::Type{DataRS}, sweep::PSweep) = ones(DataRS, PSweep[sweep])


#==Data generation
===============================================================================#
function _ensuresweepunique(d::DataRS, sweepid::String)
Expand Down
77 changes: 77 additions & 0 deletions src/doc.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#EasyPlot: docstring definitions
#-------------------------------------------------------------------------------

#==
===============================================================================#

@doc """`fill(fn::Function d::DataRS, sweeplist)`
Construct a DataRS structure storing results from parametric sweeps using recursive data structures.
# Examples
```julia-repl
signal = fill(DataRS, PSweep("A", [1, 2, 4] .* 1e-3)) do A
fill(DataRS, PSweep("phi", [0, 0.5, 1] .* (π/4))) do 𝜙
fill(DataRS{DataF1}, PSweep("freq", [1, 4, 16] .* 1e3)) do 𝑓
𝜔 = 2π*𝑓; T = 1/𝑓
Δt = T/100 #Define resolution from # of samples per period
Tsim = 4T #Simulated time
t = DataF1(0:Δt:Tsim) #DataF1 creates a t:{y, x} container with y == x
sig = A * sin(𝜔*t + 𝜙) #Still a DataF1 sig:{y, x=t} container
return sig
end; end; end
```
Note that inner-most sweep needs to specify element type (DataF1).
Other (scalar) element types include: DataInt/DataFloat/DataComplex.
""" Base.fill(::Function, ::Type{DataRS}, args...)

@doc """`fill(value, [DataRS/DataF1], sweeplist)`
Construct a filled DataRS or DataHR structure with the provided list of parametric sweeps.
# Examples
```julia-repl
sweeplist = PSweep[
PSweep("A", [1, 2, 4])
PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)
PSweep("phi", π .* [1/4, 1/3, 1/2])
]
all32s = fill(32.0, DataRS, sweeplist)
```
""" Base.fill(::DF1_Num, ::Type{DataMD}, args...)

@doc """`zeros([DataRS/DataF1], sweeplist)`
Construct a 0.0-filled DataRS or DataHR structure with the provided list of parametric sweeps.
# Examples
```julia-repl
sweeplist = PSweep[
PSweep("A", [1, 2, 4])
PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)
PSweep("phi", π .* [1/4, 1/3, 1/2])
]
all0s = zeros(DataRS, sweeplist)
```
""" Base.zeros(::Type{DataMD}, args...)

@doc """`ones([DataRS/DataF1], sweeplist)`
Construct a 1.0-filled DataRS or DataHR structure with the provided list of parametric sweeps.
# Examples
```julia-repl
sweeplist = PSweep[
PSweep("A", [1, 2, 4])
PSweep("freq", [1, 2, 4, 8, 16] .* 1e3)
PSweep("phi", π .* [1/4, 1/3, 1/2])
]
all1s = ones(DataRS, sweeplist)
```
""" Base.ones(::Type{DataMD}, args...)

#Last line
50 changes: 34 additions & 16 deletions test/datahr.jl
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,35 @@ sweeplist3 = PSweep[ #Points of 2nd sweep are different than sweeplist1
PSweep("v2", [3,4])
]

dhr1 = DataHR(sweeplist1,DataF1[d1 d1; d1 d1])
dhr2 = DataHR(sweeplist2,DataF1[d1 d1; d1 d1])
dhr3 = DataHR(sweeplist3,DataF1[d1 d1; d1 d1])
#Construct using low-level functions:
dhr1_ll = DataHR(sweeplist1,DataF1[d1 d1; d1 d1])

dhr1 = fill(d1, DataHR, sweeplist1)
dhr2 = fill(d1, DataHR, sweeplist2)
dhr3 = fill(d1, DataHR, sweeplist3)


#==Tests
===============================================================================#
@testset "Validate _datamatch()" begin show_testset_description()
other = dhr1*1.0
@test !_datamatch(dhr1, other+1) #Make sure _datamatch works
@test _datamatch(dhr1, other)
end

@testset "Construction of DataHR objects" begin show_testset_description()
dhr1_0 = zeros(DataHR, sweeplist1)
dhr1_1 = ones(DataHR, sweeplist1)
dhr1_2 = fill(2, DataHR, sweeplist1)

@test _datamatch(dhr1_2-dhr1_1, dhr1_1)
@test _datamatch(dhr1_2-dhr1_1, dhr1_1)
@test _datamatch(dhr1_2-2*dhr1_1, dhr1_0)

@test _datamatch(dhr1_0.elem, zeros(Float64, size(DataHR, sweeplist1)))
@test _datamatch(dhr1_1.elem, ones(Float64, size(DataHR, sweeplist1)))
end

@testset "Broadcast operations on DataHR" begin show_testset_description()
s1 = dhr1 + dhr1
@testset "Validate s1 = dhr1+dhr2" begin
Expand All @@ -40,21 +62,17 @@ dhr3 = DataHR(sweeplist3,DataF1[d1 d1; d1 d1])
@test_throws ArgumentError dhr1+dhr3 #Mismatched sweeps
end

@testset "DataHR => DataRS conversion" begin show_testset_description()
drs1 = DataRS(dhr1)
@testset "Conversion (DataRS <=> DataHR)" begin show_testset_description()
datahr_1 = dhr1
datars = convert(DataRS, datahr_1)
datahr_2 = convert(DataHR, datars)
@test _datamatch(datahr_1, datahr_2)

#Verify that the sweeps match:
@test drs1.sweep == dhr1.sweeps[1]
@test drs1.elem[1].sweep == dhr1.sweeps[2]
@test drs1.elem[2].sweep == dhr1.sweeps[2]

#Verify that each element of the two data structures match:
a = dhr1.sweeps
for i in 1:length(a[1]), j in 1:length(a[2])
_drs = drs1.elem[i].elem[j]; _dhr = dhr1.elem[i,j]
@test _drs.x == _dhr.x
@test _drs.y == _dhr.y
end
@test datars.sweep == dhr1.sweeps[1]
@test datars.elem[1].sweep == datahr_1.sweeps[2]
@test datars.elem[2].sweep == datahr_1.sweeps[2]

end

end
Loading

2 comments on commit 449b451

@ma-laforge
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register

Release notes:
Add more fill() constructor functions.

Also:
-Improve coverage.
-Improve docs.
-Add ones() and zeros() constructor functions.
-Bump to v0.3.4.

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/31259

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v0.3.4 -m "<description of version>" 449b451619371143e52655bce57cb7335b5b3a34
git push origin v0.3.4

Please sign in to comment.