Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for importing projects from other sites #8

Open
anishathalye opened this issue Jun 26, 2016 · 29 comments
Open

Add support for importing projects from other sites #8

anishathalye opened this issue Jun 26, 2016 · 29 comments
Labels
enhancement medium priority

Comments

@anishathalye
Copy link
Owner

It could be cool to support import from Devpost. Does Devpost have an API? I haven't looked into this yet.
@nealrs
Copy link

nealrs commented Jun 27, 2016

@anishathalye nope - there's no API for grabbing project/hackathon data. but there is an export function available to hackathon managers, which will give you a full export of data (project / creator info, prizes applied for, name, link, answers to custom submission form questions etc.) That's the data used for our expo app.

a project API would certainly be nice, but it's not in plan right now.

let me know how @devpost (well, how i) can help. i've been interested in building an expo judging app for a long time now.

@anishathalye
Copy link
Owner Author

Having a section in the readme about how to go from Devpost project export -> judging system would be nice.

Based on a quick read over the docs for the expo app, it looks like it could be accomplished by doing a project export (as CSV), having a script rearrange stuff in the CSV so it's in the right format for Gavel, and then copy-pasting the whole thing into the judging system.

@nealrs
Copy link

nealrs commented Jun 27, 2016

Sure, that'd work. Are you planning expand functionality for more
complicated judging scenarios?

Pennapps/mhacks both have categoryies / routes to that they use to assign
judges / figure out their top stage projects.

On Monday, June 27, 2016, Anish Athalye notifications@github.com wrote:

Having a section in the readme about how to go from Devpost project export
-> judging system would be nice.

Based on a quick read over the docs for the expo app, it looks like it
could be accomplished by doing a project export (as CSV), having a script
rearrange stuff in the CSV so it's in the right format for Gavel, and then
copy-pasting the whole thing into the judging system.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#8 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABftvJgUjT6FrvRkYx5-5n-BC8vaqhAIks5qP-dIgaJpZM4I-e9L
.

Neal Shyam http://nealshyam.com
nealrs@gmail.com
twitter https://twitter.com/nealrs | devpost http://devpost.com/nealrs
| github https://github.com/nealrs

@anishathalye
Copy link
Owner Author

Probably eventually. When it happens depends on whether people are interested in contributing those features. HackMIT gives judges guidelines but we only collect binary information rather than have judges compare projects across different categories (and our method has worked pretty well for us).

@nealrs
Copy link

nealrs commented Jun 27, 2016

I suppose you'd want to use the data from the expo app actually, not the
hackathon - which has a lot of irrelevant data and no table / location
numbers ( which have to be assigned manually anyway).

On Monday, June 27, 2016, Anish Athalye notifications@github.com wrote:

Probably eventually. When it happens depends on whether people are
interested in contributing those features. HackMIT gives judges guidelines
but we only collect binary information rather than have judges compare
projects across different categories (and our method has worked pretty well
for us).

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#8 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABftvIKQQZhZ82oa7pEoltDGHnnpJ9TFks5qP-4SgaJpZM4I-e9L
.

Neal Shyam http://nealshyam.com
nealrs@gmail.com
twitter https://twitter.com/nealrs | devpost http://devpost.com/nealrs
| github https://github.com/nealrs

@anishathalye
Copy link
Owner Author

Hmm, the way we do it at HackMIT, people choose table numbers themselves and submit their table number as part of their project submission.

@nealrs
Copy link

nealrs commented Jun 27, 2016

yeah - i know yall do it a little differently. in my experience, organizers like to balance our their expos manually (because of categories mentioned above / 2 expos / etc.) i think it makes a lot more sense for organizers to set table numbers, because then you (the organizer) are the one safe source for that data.

anyway, i'll try to write a python script to reformat the expo app CSV - but you might want to consider building that & file uploads into the product instead of copying & pasting.

I tried messing around with handsontable a few times, powerful stuff / super annoying to code with.

@anishathalye
Copy link
Owner Author

Yeah, it would be good to have more flexibility in the judging system to allow different styles, etc.

But yeah, building stuff into the app would be nice... it's on the roadmap, and it'll probably happen eventually.

@anishathalye anishathalye mentioned this issue Jul 27, 2016
Closed
@aMiss-aWry
Copy link

I'll look into this. Self-assigning.

@aMiss-aWry aMiss-aWry self-assigned this Jul 27, 2016
@anishathalye anishathalye mentioned this issue Feb 8, 2017
Open
@revalo
Copy link
Contributor

revalo commented Feb 23, 2017

Working on this. Have a hacky system functional, will need some heavy refactoring though.

@murby
Copy link

murby commented Feb 23, 2017

@revalo - Richard from Devpost here. Let me know if you have any questions about the Devpost data or exports and I can help. I'm here and richard@devpost.com

@anishathalye
Copy link
Owner Author

Also, it looks like HackRU has done something with Devpost integration: https://github.com/HackRU/GavelRU/commit/3fd5d85433c2074d2a94a4ac52519953ff9a27f8 (I haven't looked into this in detail though)

@nealrs
Copy link

nealrs commented Feb 23, 2017 via email

@anishathalye
Copy link
Owner Author

Does Devpost have an API?

@nealrs
Copy link

nealrs commented Feb 23, 2017 via email

@murby
Copy link

murby commented Feb 23, 2017

Neal is right - in general it's safer to use the exports as we would provide a lot of notice before making any changes there.

We are also working on some changes to the /submissions page - to improve mobile experience and expo workflow (i.e. table numbers / sponsor prizes). It's possible we could get something done JSON wise as part of that set of work. Would that be helpful?

@anishathalye
Copy link
Owner Author

Yeah, I think having a reasonable API for getting [(name, description, location)] would be helpful.

I wonder how @revalo is accessing the data in his implementation

@murby
Copy link

murby commented Feb 23, 2017

Agreed - I think there are a few other elements of the submission which might be helpful; team members, selected sponsor prizes, etc. Let's see what @revalo has done, but I'll flag this to our product and dev teams tomorrow as I know they are thinking about that page.

@nealrs
Copy link

nealrs commented Feb 23, 2017 via email

@anishathalye
Copy link
Owner Author

Why not have people be able to report the table numbers they are currently at, instead of assigning new table numbers?

@nealrs
Copy link

nealrs commented Mar 5, 2017 via email

@anishathalye
Copy link
Owner Author

Well, this could also be caused by people using tools that don't support table number submission -- if it's not possible to collect table numbers in-band, organizers resort to assigning them after the fact (because it's a huge pain to do Devpost + a google form or something for table number submission).

I think in many cases, it doesn't make sense to reassign table numbers. At HackMIT and Blueprint, we number the tables anyways (so we can find people when using HELPq), and we have people submit those same table numbers in preparation for judging. It works wonderfully for our events.

cc @andrewilyas

@nealrs
Copy link

nealrs commented Mar 6, 2017 via email

@andrewilyas
Copy link

  1. This might be a case of circular reasoning---Devpost thinks hackathons want to assign new table numbers and move people around (which, firsthand, is not great UX on either end), and so the option to input table numbers is omitted; meanwhile, hackathons want to use Devpost, and so they're forced to stick to the paradigm of reassigning table numbers, which in turn makes Devpost think that's the preferred model.

  2. Even if you disagree with (1), is there any harm in just having a Yes/No option for hackathon organizers to include a "Table Number" field? If hackathons actually prefer reassigning, they can leave this field out, whereas hackathons who prefer not to have hackers moving around right before judging can include the field in the CSV export.

@murby
Copy link

murby commented Mar 6, 2017 •
edited
Loading

@andrewilyas

Just to be 100% clear - we have never made any decision to enforce any 'preferred model', as we simply don't have one. We focus on make the tool helpful to the largest possible array of workflows. Including, in this case, teams reporting or organizers assigning table numbers.

It's simple for hackathon managers to a "table number" field to their Devpost submission form, takes two seconds. When they are setting up the site they have the ability to add a custom question which could easily be "Table Number". This would then be reflected as a column in the export we're discussing here. It can even be added once the event is live and underway if the organizers change their mind about how things are going to run.

@YasserDRIF
Copy link

Any news on this great project supporting Devpost import or somekind of API

@anishathalye
Copy link
Owner Author

I'm not personally working on this; I can check in with the HackMIT team to see if anyone is interested in implementing this feature.

@YasserDRIF
Copy link

@anishathalye yeah that would be cool

@rk9109
Copy link

rk9109 commented Feb 27, 2019 •
edited
Loading

We used a short python script this year at Blueprint to convert the devpost CSV output to a format for Gavel.

As @murby mentioned, the location field needs to be included as a custom question in the devpost submission form. The name of the field in the devpost CSV depends on the name of the custom question.

I believe this script could be cleaned up and integrated easily with Gavel, although the name of the location field would need to be specified manually. Not sure if this is the most elegant way to go about resolving this issue, though.

The script:

import csv
import argparse
import pandas as pd

def convert(file_in, file_out, columns):
    """
    Processes devpost output for gavel

    Args: file_in  | input devpost csv
          file_out | output devpost csv
          columns  | columns in file_in to select
    """
    # devpost output has variable number of columns
    # use csv.reader to clean the file first
    csv_file = open(file_in)
    csv_lines = list(csv.reader(csv_file))
    header, values = csv_lines[0], csv_lines[1:]
    data_in = {h:v for h, v in zip(header, zip(*values))}

    # select columns in data_in based on input columns
    data_out = {key:data_in[key] for key in columns}	
	
    df = pd.DataFrame.from_dict(data_out)
    df.to_csv(file_out, header=False, index=False)	

    return df

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('-i', '--input', dest='input', help='input devpost csv')
    parser.add_argument('-o', '--output', dest='output', help='output devpost csv')	
    options = parser.parse_args()
    
    # parameters [Name, Location, Description URL, Category]
    columns = ['Submission Title', 'Table Number', 'Submission Url', 'Category']
    
    df = convert(options.input, options.output, columns)

    # print some useful information
    print('Number of submissions: ', df.shape[0])
    
    count = df[columns[3]].value_counts().to_dict()
    print('Categories: ')
    for key in count.keys():
        print('\t{}: {}'.format(key, count[key]))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement medium priority
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@murby @nealrs @anishathalye @revalo @aMiss-aWry @andrewilyas @YasserDRIF @rk9109