using opengrok/docker image for perforce repository

[kee98]

I am working on setting up new opengrok instance for our organization and as obvious my preference is using the Docker version.
I followed all the steps and able to setup the UI. Even I am able to see the perforce project under a "Project" section on the home dashboard. Search, explore repo etc works well. However, History and Annotation links are always disabled. Is this because, docker image doesn't support perforce? If so, any suggestion if we can tweak some code (patch to make it work) to make it work for perforce. I also see perforce folder not created under /opengrok/data/historycache folder (whereas for git projects, it has created the folder).

Thanks in advance.

[vladak]

Yep, Perforce is not present in the image, see https://github.com/oracle/opengrok/blob/master/Dockerfile and yes, the history/annotations are greyed out if the repository cannot provide them in any way.

Now, I am not sure whether you can actually distribute Perforce in a Docker image listed on Docker Hub.

[kee98]

The value of the org.opengrok.indexer.history.Perforce property has to be the full path to the actual p4 binary.

Thanks, it seems this was useful to some extent. Now I can see historycache created for perforce project, however it has only file OpenGroklatestRev with content commit id and following warnings in log:

WARNING: Changes missed revision 17198 for /perforce-xxx.abc/proj/main/webapps/watchbuild/views.py
May be its waiting for indexer to finish before creating full cache. I will wait and will try to check this more tomorrow.

[vladak]

What is in the indexer logs actually ?

[kee98]

I just see a lot of similar message in indexer log:

WARNING: Fields [hist] will be ignored because they were not found in index directory MMapDirectory@/opengrok/data/index/perforce-xxx.abc.stage lockFactory=org.apache.lucene.store.NativeFSLockFactory@6a6dda4b

[kee98]

Basically after manually going inside the perforce directory and running the command:
java -Dorg.opengrok.indexer.history.Perforce=/opengrok/dist/lib/ ...

it creates history direcory but only with file name OpenGroklatestRev which contains revision id. and nothing else.

[vladak]

I just see a lot of similar message in indexer log:

WARNING: Fields [hist] will be ignored because they were not found in index directory MMapDirectory@/opengrok/data/index/perforce-xxx.abc.stage lockFactory=org.apache.lucene.store.NativeFSLockFactory@6a6dda4b

This is from the webapp log, not indexer log. Also, at this point such warning is innocuous.

[vladak]

What is the return code of '/usr/bin/p4 help' ?

…

________________________________________ From: kee98 ***@***.***> Sent: Tuesday, November 8, 2022 5:46 AM To: oracle/opengrok Cc: Vladimir Kotal; Comment Subject: [External] : Re: [oracle/opengrok] using opengrok/docker image for perforce repository (Discussion #4090) I tried merging your change in my local and build the image. Now I can see following error while running this command: java -Dorg.opengrok.indexer.history.Perforce=/usr/bin/p4 -jar /opengrok/lib/opengrok.jar -H -S -P -s /opengrok/src -d /opengrok/data log: WARNING: Command '/usr/bin/p4' does not work. Some operations with Perforce repositories will fail as a result. However, /usr/bin/p4 exists: # /usr/bin/p4 info User name: name Client name: opengrok-dev-stage-xxx-02 Client host: opengrok-c5898674f-m2wvf ... ... — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/oracle/opengrok/discussions/4090*discussioncomment-4082833__;Iw!!ACWV5N9M2RV99hQ!IAGYosu7dd0gcRMtCnn-33YTV0CdJt2uf2pGBL3LgJ0NJTfYUDrPboehlOybGcRIGfD6YhrcyI7aULx4ibfvDYAm4lv2$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AAWMMDHXS4KJB6BGKLT6PB3WHHLK3ANCNFSM6AAAAAARXD4RTU__;!!ACWV5N9M2RV99hQ!IAGYosu7dd0gcRMtCnn-33YTV0CdJt2uf2pGBL3LgJ0NJTfYUDrPboehlOybGcRIGfD6YhrcyI7aULx4ibfvDf1HCwvW$>. You are receiving this because you commented.Message ID: ***@***.***>

[kee98]

If I run it from /opengrok/src/<perforce_directory/ then I get 0 return code. However, from other directory, I get 1 return code.

[vladak]

Okay, that explains the failure. Will change the Perforce check from p4 help to p4 -V as the latter seems to exit with 0 no matter in which directory it is executed.

[kee98]

this is my complete log, if that helps. It seems to be picking the directory right, but not generating history instead just creates OpenGroklatestRev file.

When I run history command for first time, I get several warning like:

PerforceHistoryParser.parseTruncatedFileLog: Changes missed revision xxxx for file <>/build.py

and then for any subsequent run it shows below logs. That means, its picking the project fine, but somehow (some configuration?) its only generating OpenGroklatestRev file and not actualy history.

Exuberant Ctags 5.8, Copyright (C) 1996-2009 Darren Hiebert
  Compiled: Nov  3 2022, 14:18:55
  URL: https://ctags.io/
  Optional compiled features: +wildcards, +regex, +iconv, +option-directory, +packcc, +optscript
13:03:22 INFO: Scanning for repositories in [/opengrok/src/perforce-xxx.abc.is.here] (down to 2 levels below source root)
13:03:22 WARNING: Command '/usr/bin/p4' does not work. Some operations with Perforce repositories will fail as a result.
13:03:22 INFO: Done scanning for repositories, found 1 repositories (took 490 ms)
13:03:22 INFO: Generating history cache for all repositories ...
13:03:23 INFO: Creating history cache for 1 repositories
13:03:23 INFO: Creating history cache for /opengrok/src/perforce-xxx.abc.is.here (PerforceRepository) without renamed file handling
13:03:23 INFO: Done history cache for /opengrok/src/perforce-xxx.abc.is.here (took 249 ms)
13:03:23 INFO: Done history cache for all repositories (took 397 ms)
13:03:23 INFO: Done generating history cache
13:03:23 INFO: Starting indexing

I get the warning for /usr/bin/p4 (even after changing boolean works = checkCmd(repoCommand, "-V"); in opengrok-indexer/src/main/java/org/opengrok/indexer/history/PerforceRepository.java file.

Surprise to see there are no good/ detailed documentations for opengrok with perforce :( . If there are any, please share (even if it is for non-docker opengrok).

[kee98]

ok, some magic happened and now I can see history and annotations...not sure what exactly I did but it was just some random different things and debugging the codes. I will try this on fresh setup and update this thread if there are anything useful for opengrok community.

[vladak]

I get the warning for /usr/bin/p4 (even after changing boolean works = checkCmd(repoCommand, "-V"); in opengrok-indexer/src/main/java/org/opengrok/indexer/history/PerforceRepository.java file.

There was logic error in the code so the warning was always displayed. Fixed that in the PR.

Surprise to see there are no good/ detailed documentations for opengrok with perforce :( . If there are any, please share (even if it is for non-docker opengrok).

There may be some hints in closed OpenGrok issues. I did some spelunking there however did not find anything valuable.