Skip to content

This project is to formalise the iPinYou RTB data into a standard format for further researches.

License

Notifications You must be signed in to change notification settings

wnzhang/make-ipinyou-data

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

make-ipinyou-data

This project is to formalise the iPinYou RTB data into a standard format for further researches.

Step 0

The raw data of iPinYou (ipinyou.contest.dataset.zip) can be downloaded from UCL website.

Unzip it and get the folder ipinyou.contest.dataset.

Step 1

Update the soft link for the folder ipinyou.contest.dataset in original-data.

weinan@ZHANG:~/Project/make-ipinyou-data/original-data$ ln -sfn ~/Data/ipinyou.contest.dataset ipinyou.contest.dataset

Under make-ipinyou-data/original-data/ipinyou.contest.dataset there should be the original dataset files like this:

weinan@ZHANG:~/Project/make-ipinyou-data/original-data/ipinyou.contest.dataset$ ls
algo.submission.demo.tar.bz2  README         testing2nd   training3rd
city.cn.txt                   region.cn.txt  testing3rd   user.profile.tags.cn.txt
city.en.txt                   region.en.txt  training1st  user.profile.tags.en.txt
files.md5                     testing1st     training2nd

You do not need to further unzip the packages in the subfolders.

Step 2

Under make-ipinyou-data folder, just run make all.

After the program finished, the total size of the folder will be 14G. The files under make-ipinyou-data should be like this:

weinan@ZHANG:~/Project/make-ipinyou-data$ ls
1458  2261  2997  3386  3476  LICENSE   mkyzxdata.sh   python     schema.txt
2259  2821  3358  3427  all   Makefile  original-data  README.md

Normally, we only do experiment for each campaign (e.g. 1458). all is just the merge of all the campaigns. You can delete all if you think it is unuseful in your experiment.

Use of the data

We use campaign 1458 as example here.

weinan@ZHANG:~/Project/make-ipinyou-data/1458$ ls
featindex.txt  test.log.txt  test.yzx.txt  train.log.txt  train.yzx.txt
  • train.log.txt and test.log.txt are the formalised string data for each row (record) in train and test. The first column is whether the user click the ad or not. The 14th column is the winning price for this auction.
  • featindex.txtmaps the features to their indexes. For example, 8:115.45.195.* 29 means that the 8th column in train.log.txt with the string 115.45.195.* maps to feature index 29.
  • train.yzx.txt and test.yzx.txt are the mapped vector data for train.log.txt and test.log.txt. The format is y:click, z:wining_price, and x:features. Such data is in the standard form as introduced in iPinYou Benchmarking.

For any questions, please report the issues or contact Weinan Zhang.

About

This project is to formalise the iPinYou RTB data into a standard format for further researches.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •