Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Html import #189

Open
wants to merge 69 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
49fccef
First version of QTI parser
rlgreen91 May 18, 2012
c337b1b
New unit test for QTI parser
rlgreen91 May 18, 2012
5dd839c
Added exceptions to the QTI parser
rlgreen91 May 18, 2012
9770932
Made the parser more flexible in terms of how it accepts inputs, and …
rlgreen91 May 21, 2012
b301335
Merge remote-tracking branch 'upstream/master' into HTML_import
rlgreen91 May 22, 2012
f44b892
Changed the parser so that it accepts almost any character that comes…
rlgreen91 May 23, 2012
72af70b
Attempting to write an execption for images within questions.
rlgreen91 May 23, 2012
8ad5b4c
Fixed the parser(again). Now, the transformer will replace HTML ital…
rlgreen91 May 25, 2012
3f1c8f1
Deleting old files
rlgreen91 May 25, 2012
1300e7b
Parser can now identify images inline in questions.
rlgreen91 May 25, 2012
9219a81
Still having problems with getting the parser to recognize images and…
rlgreen91 May 26, 2012
354d020
Finally got it to recognize and raise exceptions.
rlgreen91 May 26, 2012
5d5462b
Now replaces teletype tags with inline LaTex tags.
rlgreen91 May 26, 2012
37d9a8e
Recognizes and replaces paragraph tags with correct formatting.
rlgreen91 May 26, 2012
1334f13
Parser now detects for font changes and makes the content for those c…
rlgreen91 May 26, 2012
404f959
Renames some files. Finished the first parser.
rlgreen91 May 26, 2012
c8d12a4
Added replacements for greek letters, subscripts, superscripts, and c…
rlgreen91 May 26, 2012
e41465e
Created an import file that takes as inputs the filename and the type…
rlgreen91 May 26, 2012
c1987ff
Now, the import class will also decided which parser and transformer …
rlgreen91 May 26, 2012
3b917eb
Import class can now grab just the relevant content, and can pick out…
rlgreen91 May 26, 2012
703575f
Ok, so this is the import class. It determines what parser and trans…
rlgreen91 May 27, 2012
0c0d9c3
Ok, so now the import class can also go through the list of answer ch…
rlgreen91 May 27, 2012
f10961a
Now the import class can go through the list of relevant answer choic…
rlgreen91 May 27, 2012
d58f9b6
Redid the function for finding the points for each answer choice to s…
rlgreen91 May 27, 2012
42db907
Finished making the whole import class capable of handling any number…
rlgreen91 May 27, 2012
9620304
All of the unit test for the importer work.
rlgreen91 May 27, 2012
78796c6
The import class is done. Everything works and the structure has bee…
rlgreen91 May 27, 2012
961cb95
So, I've created a view that contains a form to enter the information…
rlgreen91 May 27, 2012
a2ad4f6
Added gem Carrierwave to handle file uploads.
rlgreen91 Jun 13, 2012
26e9bfe
All of the files associated with creating the form to upload files, c…
rlgreen91 Jun 13, 2012
fbfbb34
Updated parser to account for more hidden HTML tags.
rlgreen91 Jun 15, 2012
c84ad43
Just updated the parser to account for more tags.
rlgreen91 Jun 15, 2012
6928c3f
Updated parser to handle arbitrarily nested HTML tags.
rlgreen91 Jun 15, 2012
ccd14e6
Updating rails
rlgreen91 Jun 21, 2012
9c07b91
Upgraded to Rails 3.2
rlgreen91 Jun 21, 2012
7abbf7b
Had to redo form to be compatible with new Rails.
rlgreen91 Jun 21, 2012
e1ca277
Switching branches
rlgreen91 Jun 23, 2012
47c7aaf
Optimized the import functions, so this feature should run faster now…
rlgreen91 Jun 24, 2012
b0bacf0
More changes to optimize the import function
rlgreen91 Jun 24, 2012
69a5ecc
Made the import function a little bit faster and added more things to…
rlgreen91 Jun 26, 2012
3c61d4a
Got the whole test file to be imported correctly.
rlgreen91 Jun 27, 2012
5b20127
Added functionality to handle any problems with import. Also added a…
rlgreen91 Jun 27, 2012
8ecb0fb
cleaning up code
rlgreen91 Jun 27, 2012
40f57f9
cleaning up code
rlgreen91 Jun 27, 2012
6bf734e
uninstalled gems
rlgreen91 Jun 27, 2012
973acf1
merge conflicts
rlgreen91 Jun 27, 2012
55ed402
Improved user search by name
Dantemss Jun 11, 2012
782bd6f
finished js answer alert feature
Jul 6, 2012
6d49bff
merge conflicts
rlgreen91 Jul 18, 2012
bcfbbda
fixed problem with icon disappearing with rails update
sebrittain Jul 9, 2012
5b95532
Import function now looks for images attached to questions that are n…
rlgreen91 Jul 13, 2012
3c0b18f
Trying to get filenames from extracted zip file
rlgreen91 Jul 14, 2012
0039935
Able to unzip zip files uploaded to import function. Still have trou…
rlgreen91 Jul 17, 2012
16b3982
Cleaning up files
rlgreen91 Jul 18, 2012
2cdfeaf
Import function now looks for images attached to questions that are n…
rlgreen91 Jul 13, 2012
b295e14
Trying to get filenames from extracted zip file
rlgreen91 Jul 14, 2012
75dedfd
Able to unzip zip files uploaded to import function. Still have trou…
rlgreen91 Jul 17, 2012
68ea8cd
finished js answer alert feature
Jul 6, 2012
f1eccfd
fixed problem with icon disappearing with rails update
sebrittain Jul 9, 2012
7d22d9c
Merge remote-tracking branch 'upstream/master' into temp3
rlgreen91 Jul 18, 2012
899ec3a
Import function now unzips a file,and successfully finds the xml docu…
rlgreen91 Jul 18, 2012
4332e8e
Now import function takes a directory containing subfolders of images…
rlgreen91 Jul 18, 2012
c82499b
stopping to switch to separate issue
rlgreen91 Jul 18, 2012
03adfa2
Merge remote-tracking branch 'upstream/master' into HTML_import
rlgreen91 Jul 18, 2012
79baf64
Able to add attached images as assets. Now working on saving them as …
rlgreen91 Jul 18, 2012
887d9e8
Now the import function can auto-add images into questions
rlgreen91 Jul 19, 2012
a343cfd
Cleaning up code
rlgreen91 Jul 19, 2012
932af42
Updating unit tests
rlgreen91 Jul 21, 2012
5426a69
Updating tests
rlgreen91 Jul 21, 2012
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ end

gem 'jquery-rails', '~> 2.0.2'

gem 'nokogiri'

gem 'htmlentities'

gem 'rubyzip'

gem 'archive-zip'

# Deploy with Capistrano
gem 'capistrano', '~> 2.12.0'

Expand Down
20 changes: 15 additions & 5 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,10 @@ GEM
multi_json (~> 1.0)
acts-as-taggable-on (2.3.1)
rails (~> 3.0)
archive-zip (0.5.0)
io-like (>= 0.3.0)
arel (3.0.2)
bcrypt-ruby (3.0.1)
bcrypt-ruby (3.0.1-x86-mingw32)
blankslate (2.1.2.4)
builder (3.0.0)
bullring (0.8.2)
Expand Down Expand Up @@ -81,7 +82,9 @@ GEM
i18n (~> 0.4)
highline (1.6.13)
hike (1.2.1)
htmlentities (4.3.1)
i18n (0.6.0)
io-like (0.3.0)
journey (1.0.4)
jquery-rails (2.0.2)
railties (>= 3.2.0, < 5.0)
Expand All @@ -100,14 +103,14 @@ GEM
mime-types (1.19)
multi_json (1.3.6)
mysql2 (0.3.11)
mysql2 (0.3.11-x86-mingw32)
net-scp (1.0.4)
net-ssh (>= 1.99.1)
net-sftp (2.0.5)
net-ssh (>= 2.0.9)
net-ssh (2.5.2)
net-ssh-gateway (1.1.0)
net-ssh (>= 1.99.1)
nokogiri (1.5.4)
orm_adapter (0.3.0)
paperclip (3.0.4)
activemodel (>= 3.0.0)
Expand Down Expand Up @@ -151,6 +154,7 @@ GEM
rdoc (3.12)
json (~> 1.4)
recaptcha (0.3.4)
rubyzip (0.9.9)
sass (3.1.19)
sass-rails (3.2.5)
railties (~> 3.2.0)
Expand All @@ -163,7 +167,6 @@ GEM
rack (~> 1.0)
tilt (~> 1.1, != 1.3.0)
sqlite3 (1.3.6)
sqlite3 (1.3.6-x86-mingw32)
squeel (1.0.6)
activerecord (~> 3.0)
activesupport (~> 3.0)
Expand All @@ -182,21 +185,25 @@ GEM
polyglot
polyglot (>= 0.3.1)
tzinfo (0.3.33)
uglifier (1.2.5)
execjs (>= 0.3.0)
multi_json (~> 1.3)
uglifier (1.2.6)
execjs (>= 0.3.0)
multi_json (~> 1.3)
upmark (0.1.3)
parslet
warden (1.2.1)
rack (>= 1.0)
will_paginate (3.0.3)
yajl-ruby (1.1.0)
yajl-ruby (1.1.0-x86-mingw32)

PLATFORMS
ruby
x86-mingw32

DEPENDENCIES
acts-as-taggable-on (~> 2.3.1)
archive-zip
bullring (~> 0.8.2)
capistrano (~> 2.12.0)
coffee-rails (~> 3.2.2)
Expand All @@ -205,18 +212,21 @@ DEPENDENCIES
execjs (~> 1.4.0)
factory_girl_rails (~> 3.4.0)
faker (~> 1.0.1)
htmlentities
jquery-rails (~> 2.0.2)
jsonify (~> 0.3.1)
jsonify-rails (~> 0.3.2)
mime-types (~> 1.18)
mysql2 (~> 0.3.11)
nokogiri
paperclip (~> 3.0.4)
parslet (~> 1.4.0)
quiet_assets (~> 1.0.1)
rack-ssl-enforcer (~> 0.2.4)
rails (= 3.2.4)
rails3-jquery-autocomplete (~> 1.0.7)
recaptcha (~> 0.3.4)
rubyzip
sass-rails (~> 3.2.5)
single_test (~> 0.5.1)
sqlite3 (~> 1.3.6)
Expand Down
2 changes: 2 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#encoding: utf-8

# Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details.

Expand Down
5 changes: 5 additions & 0 deletions app/controllers/import_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class ImportController < ApplicationController
def index
@qt_import = QtImport.new
end
end
30 changes: 30 additions & 0 deletions app/controllers/qt_import_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#encoding: utf-8

# Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details.

require 'mime/types'

class QtImportController < ApplicationController
def new
@content_types = QTImport.content_types
end

def create
begin
a = params[:file]
a,images = QTImport.unzip(a.path,"#{Rails.root}/tmp/import") if a.content_type == 'application/zip'
document = QTImport.openfile(a.path)
project = QTImport.createproject(current_user)
parser, transformer = QTImport.choose_import(params[:content_type])
content = QTImport.iterate_items(document)
QTImport.get_questions(project,content,parser,transformer,current_user) unless images
QTImport.get_questions(project,content,parser,transformer,current_user,images) unless !images

rescue
flash[:alert] = 'Unfortunately we could not import all of your questions. This may be due to formatting errors in your questions, such as HTML.
However, we were most likely able to import some of your questions. They are listed under a new project called Import, and are now unpublished questions.'
redirect_to :action => :new
end
end
end
5 changes: 5 additions & 0 deletions app/helpers/qt_import_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details.

module QtImportHelper
end
16 changes: 16 additions & 0 deletions app/views/qt_import/_form.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<!-- # Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details. -->

<%= form_tag("/import/qti", :multipart => true) do %>
<%= label_tag("File") %>
<br>
<%= file_field_tag(:file) %>
<br>
<br>
<%= label_tag("Content Type") %>
<br>
<%= select_tag(:content_type, options_for_select(@content_types).html_safe ) %>
<br>
<br>
<%= submit_tag("Import") %>
<% end %>
6 changes: 6 additions & 0 deletions app/views/qt_import/create.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<!-- # Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details. -->

<p>Your questions were successfully imported!
<br>
They have been added to a new project named "Import", and are now unpublished questions.</p>
3 changes: 3 additions & 0 deletions app/views/qt_import/error.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<p>Unfortunately we could not import all of your questions. This may be due to formatting errors in your questions, such as HTML.
However, we were most likely able to import <i>some</i> of your questions. They are listed under a new project called Import, and
are now unpublished questions.</p>
4 changes: 4 additions & 0 deletions app/views/qt_import/index.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<h1>Import Questions</h1>
<p> Note: All files uploaded MUST be XML files. </p>

<%= render 'form' %>
7 changes: 7 additions & 0 deletions app/views/qt_import/new.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<!-- # Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details. -->

<h1>Import Questions</h1>
<p> Note: All files uploaded MUST be XML files. </p>

<%= render 'form' %>
7 changes: 7 additions & 0 deletions app/views/questions/new.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,10 @@ Use this type for simple free-form questions that may or may not have a multiple


</table>

<br>
<big>
<b>
<%= link_to "Or click this link to import a file of questions.", import_qti_new_path %>
</b>
</big>
2 changes: 2 additions & 0 deletions config/application.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#encoding: utf-8

# Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details.

Expand Down
7 changes: 7 additions & 0 deletions config/environment.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#encoding: utf-8

# Copyright 2011-2012 Rice University. Licensed under the Affero General Public
# License version 3 or later. See the COPYRIGHT file for details.

Expand All @@ -14,8 +16,13 @@
require 'form_builder_extensions'
require 'acts_as_numberable'
require 'variated_content_html'
require 'spqr_parser'
require 'qti_import'


ActionMailer::Base.delivery_method = :sendmail

Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8
# Initialize the rails application
Quadbase::Application.initialize!
3 changes: 3 additions & 0 deletions config/routes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@

Quadbase::Application.routes.draw do

get "/import/qti/new", :to => 'qt_import#new'
post "/import/qti", :to => 'qt_import#create'

namespace :admin do
resources :logic_libraries do
resources :logic_library_versions, :shallow => true
Expand Down
12 changes: 12 additions & 0 deletions db/migrate/20120613190208_create_file_uploads.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
class CreateFileUploads < ActiveRecord::Migration
def self.up
create_table :file_uploads do |t|

t.timestamps
end
end

def self.down
drop_table :file_uploads
end
end
Loading