-
Notifications
You must be signed in to change notification settings - Fork 80
Google Summer of Code 2014 Ideas
Feel free to reach us by joining #sciruby on chat.freenode.net or via our mailing list.
We strongly recommend that you pick one of the ideas listed below. We value contributions in advance of GSoC, even if they're just little ones. Go pick out something in one of our trackers and work on it, talk to folks on the listserv, and get an idea for what features are needed.
You don't need to know a lot about Ruby to work on a project: depending on how much you already know, it'll be pretty easy to learn enough to be able to contribute. However, you may need some familiarity with scientific computation. If you don't have any, take a look at "Numerical Recipes in C", which you'll probably find in your university's library.
In any case, if you feel your skills aren't enough for some project, please ask us on our IRC channel (see contact section above) and we can help you.
Our number-one priority right now as an organization is NMatrix. Our number-two priority is most likely Minimization. We need both of these libraries for some more complicated statistical methods, some of which were actually written in pure Ruby in last year's Summer of Code; we want to speed up those methods, and to do so, we need a framework that can make use of pure C and Java facilities (which Minimization can't currently do).
NMatrix is SciRuby's numerical matrix core, implementing dense matrices as well as two types of sparse (linked-list-based and Yale/CSR). NMatrix is a fairly new but well-established project which has received Summer-of-Code-like grants from both Brighter Planet and the Ruby Association (in other words, from Matz, who created Ruby). Those who contribute to NMatrix will likely eventually become authors of a jointly-published peer-reviewed science article on the library. Additionally, NMatrix is a good place to gain practical C and C++ experience, while also working to improve Ruby.
- Mentors: John Woods (@mohawkjohn)
-
ATLAS Functionality. NMatrix has many but not all ATLAS (cBLAS) and LAPACK functions exposed. We would like to see a consistent interface which makes sense in Ruby. We also want to be able to design and implement several
NMatrixmethods which depend upon ATLAS, cBLAS, and cLAPACK functions.- Rational Functionality. NMatrix includes some rational number capability, but support is lacking in areas where ATLAS functions are required, since ATLAS does not have a rational type. Rational-specific equivalents of ATLAS functions are needed. Along the way it may be possible to also implement some integer-specific ATLAS function equivalents. (This is a component of the ATLAS Functionality project, but could be proposed separately with sufficient justification.)
-
ATLAS-Free Support. NMatrix has some non-ATLAS versions of functions like
gemm(matrix multiplication) which are typically used for rational matrices. Since we have to go to the trouble of implementing rational versions of many ATLAS functions, it might be useful to also have more simplistic ATLAS-free complex and floating-point versions of those ATLAS functions as well -- primarily for those who don't have LAPACK. They wouldn't be as heavily optimized, but would serve in a tight spot. - Basic matrix math functionality. Specifically, exponentials and square roots, matrix decomposition/factorization, calculation of norms, tensor products, principal component analysis (PCA). This is listed as a sub-project of ATLAS functionality, because several depend upon ATLAS, but it could really be proposed as a separate GSOC project. These functions are all enormously important, and would substantially improve the usability of NMatrix. Successful implementation would likely lead to co-authorship on a peer-reviewed article, and at the very least would look outstanding on a curriculum vitae.
- Sparse improvements. The "new" Yale matrices used by NMatrix, which store diagonals (zero and non-zero) separately from non-diagonal non-zeros, are inefficient for matrices that are taller than they are wide. One way to address the problem would be to introduce an alternate "old" Yale storage. Another would be to allow matrices to be stored and operated on transposed. The goal, overall, is to be able to produce efficient Yale/sparse vectors regardless of the vectors' orientation.
- Mentors: Carlos Agarie (@agarie), Claudio Bustos(@clbustos), John Prince (@jtprince)
- SciRuby::Dataframe will be an implementation of a concept similar to Pandas (http://pandas.pydata.org/pandas-docs/dev/), which provides (Dataframes, tabular structures like data.frame in R and Series, for 1-dimensional data) usable by more powerful data analysis packages.
- Some requirements:
- Have some simple statistics built-in (statsample as a dependency): averages, quartiles, standard deviation, median, etc.
- Be really easy to plot. For example, a user should be able to plot a histogram from a Series object with only one method call, maybe two, without much hassle. Integration with Plotrb (see below) would be great.
- Easily receive and interpret data from a CSV file (or any delimeter separated value file), transforming it into a Dataframe with something as simple as
SciRuby::Dataframe.csv("data.csv"). This would be a wrapper around Ruby'sCSVclass. Also, chunk processing of CSV files will be necessary. Thefaster_csvgem implements this. - Be able to add/remove columns and do operations on rows or columns. For simple operations, this should be very very easy by using NMatrix's referenced slices.
- Have labeled columns and indexed rows. This means that the underlying data structure (NMatrix wrapper) will need to store some metadata.
- Use NMatrix for data storage. This also implies that we can use the NMatrix::IO module.
- As some of the requirements of this project depend on others (visualization, statistics, etc), the most important part is to design and develop it in such a way for its API to be easy to use for new users (e.g. scientists without much programming background) but extensible enough for other projects to use it.
- Inspired by Pandas and Statsample::Dataset.
- Mentors: Pjotr Prins, Rob Syme, John Woods
- D3 is an interactive plotting system for the browser. GSoC 2013 delivered static plotting through VEGA, a JSON style description of a figure. Having learnt from that project we want to create a more advanced system where Ruby generates JavaScript and JavaScript can be mixed with Ruby. The main challenge is to get from SciRuby data structures to D3 plots in the nicest possible way and still allow for the flexibility and interactivity that D3 allows.
- Requirements: This is an advanced project that requires use of powerful Ruby paradigms and a deep understanding of D3
- See also plotrb
- Mentors: John Woods (@mohawkjohn), Wan Zuhao Ruby needs a science notebook and instant plotting solution. Two examples of such pieces of software are iPython notebook and Mathematica. Ruby already has useful plotting tools, including Plotrb and Rubyvis, the first of which was a GSoC 2013 project by Wan Zuhao. Both of Plotrb and Rubyvis produce publication-quality graphics. However, sometimes a researcher doesn't need publication quality; she needs a quick-and-dirty prototype. This is where interactive notebooks come in handy. Work on a Ruby notebook might also necessitate development of some additional plotting software, but is primarily a user interface and user experience project. This project is code named "Redbook."
- Mentors: Claudio Bustos (@clbustos), John Woods for statistical distributions (@mohawkjohn)
- Statsample is an essential scientific library which brings statistical functions to Ruby. Currently, it depends upon Ruby/GSL, which conflicts with NMatrix. To bring it up to spec, it needs to require the SciRuby fork of rb-gsl instead. Lastly, Statsample depends upon Distribution, which makes available statistical distribution functions for users of MRI (in pure Ruby and through GSL) and JRuby. Many of these functions remain unimplemented, or need a JRuby or GSL or pure Ruby version written.
- There were two projects last year which became statsample extensions (as gems), statsample-timeseries and statsample-glm.
Minimization and Integration are two SciRuby modules which are used by Claudio Bustos' statsample gem. For Minimization, students would research and suggest additional minimization methods, develop tests, and improve documentation. For Integration, students would implement additional numerical integration methods and add support for solving various types of (ordinary and/or partial) differential equations. We need to be explicit about the imprecisions and performance of each method, so benchmarks will be necessary. As always, the student is expected to write tests and document code. There has been some talk of removing support for Ruby versions earlier than 1.9.3 for both Integration and Minimization.
- Mentors: Claudio Bustos (@clbustos), John Woods (@mohawkjohn)
- Standardized minimization framework. Right now, Minimization is pure Ruby. There are additional minimization algorithms in GSL and probably in Java which can be used in Ruby. It'd be great to have a standardized framework so that pure Ruby functions can be used, or C functions if using MRI/YARV, or Java functions if using JRuby. Such a thing is already done for the Distribution gem, so that can be used as a model.
- Mentors: Pjotr Prins, Toshiaki Katayama, Will Strinz, Mark Wilkinson, Jerven Bolleman
- Interactive scientific tools tend to manage complex state in RAM and allows persistence after an analysis session. One example for statistics is R and its environment. The downside of this approach is that the amount of data handled is limited by memory size of the machine. We propose to use a semantic web data store (a server or a linked library) as a generic backend for interactive data analysis and session persistence in interactive Ruby. The use of a flexible data store that allows complex objects and querying using SPARQL is appealing. Applications in scientific domains, such as bioinformatics, would especially be gratifying. The bioinformatics community is doing a lot of work integrating different data repositories. Bio2RDF contains a wide range of mapped identifiers and SADI, which is the service discovery for when data is put online. A list of activities can be found here. BioRuby and biogems contain a wide range of parsers and formatters which could be extended to support reading and writing RDF (RDF is the standard of semantic web data). Having such functionality would make it easy for bioinformaticians to incorporate and expose RDF for flexible data queries.
- Approach: We will visit all existing classes, parsers and formatters and decide which ones are most useful for RDF import/export. The student will tackle one transformer at a time, writing tests and adding a SPARQL end point for others to use. The student will also add SADI service discovery. We take bioinformatics as the applied domain, still the software should be generically useful for statistical data mining and discovery
- Difficulty and needed skills: Average difficulty
- The student will need to have affinity with the semantic web and get to a decent level op Ruby programming. Probably includes meta-programming.