Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems discovered while running the demo #166

Closed
wangxilong1991 opened this issue Jun 13, 2018 · 0 comments
Closed

Problems discovered while running the demo #166

wangxilong1991 opened this issue Jun 13, 2018 · 0 comments

Comments

@wangxilong1991
Copy link

excuse me, When I was in the cloud demo, I found the program stuck, and then I looked at the source code for RankLib-2.8.jar. Modified the LambdaMART init: look at ******

         //   sortedIdx = new int[features.length][];                 //source   ******
	sortedIdx = new int[features.length][martSamples.length];//modified by me  *****
	MyThreadPool p = MyThreadPool.getInstance();
	if(p.size() == 1)//single-thread
		sortSamplesByFeature(0, features.length-1);
	else//multi-thread
	{
		int[] partition = p.partition(features.length);
		for(int i=0;i<partition.length-1;i++)
			p.execute(new SortWorker(this, partition[i], partition[i+1]-1));
		p.await();
	}
	
	//Create a table of candidate thresholds (for each feature). Later on, we will select the best tree split from these candidates 
	thresholds = new float[features.length][];
	for(int f=0;f<features.length;f++)
	{
		//For this feature, keep track of the list of unique values and the max/min 
		List<Float> values = new ArrayList<Float>();
		float fmax = Float.NEGATIVE_INFINITY;
		float fmin = Float.MAX_VALUE;
		for(int i=0;i<martSamples.length;i++)
		{	// null point exception ****** Null pointer caused by uninitialized sortIndex
			int k = sortedIdx[f][i];//get samples sorted with respect to this feature
			float fv = martSamples[k].getFeatureValue(features[f]);
			values.add(fv);
			if(fmax < fv)
				fmax = fv;
			if(fmin > fv)
				fmin = fv;
			//skip all samples with the same feature value
			int j=i+1;
			while(j < martSamples.length)
			{
				if(martSamples[sortedIdx[f][j]].getFeatureValue(features[f]) > fv)
					break;
				j++;
			}
			i = j-1;//[i, j] gives the range of samples with the same feature value
		}

Repackage the program to run successfully !!!

Have you found this problem? Is there any other problem that I solve this way?Does ranklib have this bug?


[root@master ~]# java -jar RankLib-2.8.jar -ranker 6 -train sample_judgments_wfeatures.txt -save model.txt -frate 1.0

Discard orig. features
Training data: sample_judgments_wfeatures.txt
Feature vector representation: Dense.
Ranking method: LambdaMART
Feature description file: Unspecified. All features will be used.
Train metric: ERR@10
Test metric: ERR@10
Highest relevance label (to compute ERR): 4
Feature normalization: No
Model file: model.txt

[+] LambdaMART's Parameters:
No. of trees: 1000
No. of leaves: 10
No. of threshold candidates: 256
Min leaf support: 1
Learning rate: 0.1
Stop early: 100 rounds without performance gain on validation data

Reading feature file [sample_judgments_wfeatures.txt]... [Done.]
(3 ranked lists, 25 entries read)
Initializing... Exception in thread "pool-1-thread-1" Exception in thread "main" Exception in thread "pool-1-thread-2" ciir.umass.edu.utilities.RankLibError: Error in DenseDataPoint::getFeatureValue(): requesting unspecified feature, fid=2
at ciir.umass.edu.utilities.RankLibError.create(RankLibError.java:26)
at ciir.umass.edu.learning.DenseDataPoint.getFeatureValue(DenseDataPoint.java:26)
at ciir.umass.edu.learning.tree.LambdaMART.sortSamplesByFeature(LambdaMART.java:459)
at ciir.umass.edu.learning.tree.LambdaMART.sortSamplesByFeature(LambdaMART.java:567)
at ciir.umass.edu.learning.tree.LambdaMART$SortWorker.run(LambdaMART.java:585)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
java.lang.NullPointerException
at ciir.umass.edu.learning.tree.LambdaMART.init(LambdaMART.java:121)
at ciir.umass.edu.learning.RankerTrainer.train(RankerTrainer.java:42)
at ciir.umass.edu.eval.Evaluator.evaluate(Evaluator.java:733)
at ciir.umass.edu.eval.Evaluator.main(Evaluator.java:503)
ciir.umass.edu.utilities.RankLibError: Error in DenseDataPoint::getFeatureValue(): requesting unspecified feature, fid=1
at ciir.umass.edu.utilities.RankLibError.create(RankLibError.java:26)
at ciir.umass.edu.learning.DenseDataPoint.getFeatureValue(DenseDataPoint.java:26)
at ciir.umass.edu.learning.tree.LambdaMART.sortSamplesByFeature(LambdaMART.java:459)
at ciir.umass.edu.learning.tree.LambdaMART.sortSamplesByFeature(LambdaMART.java:567)
at ciir.umass.edu.learning.tree.LambdaMART$SortWorker.run(LambdaMART.java:585)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant