feat: Fiat-Shamir transcript using a short hash

[ivokub]

Description

Implement gnark-crypto compatible Fiat-Shamir transcript using a short MiMC hash for proof recursion.

All tests seem to be working, but I'm not sure about the option WithDomainSeparation. I think it is too agressive on gnark-crypto side to enforce domain separation with the string: string when hasher has WriteString method. I would make it optional as in gnark. Imo right now the behaviour is not documented and quite difficult to debug.

Related #847

Type of change

• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

How has this been tested?

• Transcript test with 2-chains
• Transcript test with field emulation

How has this been benchmarked?

Not benchmarked

Checklist:

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I did not modify files generated from templates
• golangci-lint does not output errors locally
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[ivokub]

One existing Fiat-Shamir test still failing, looking into it.

[github-actions]

📦 github.com/consensys/gnark/std/commitments/fri
❌ TestFriVerification 410ms

    fri_test.go:128: [assertIsEqual] 6220301374620014499393365381716476832336311931410495424048226281586478661613 == 10176871833167403566105618975605765056524646065533410792823405331921234170134
        merkle.(*MerkleProof).VerifyProof
        	verify.go:106
        fri.RadixTwoFri.verifyProofOfProximitySingleRound
        	fri.go:181
        fri.RadixTwoFri.VerifyProofOfProximity
        	fri.go:231
        fri.(*ProofOfProximityTest).Define
        	fri_test.go:40
        

📦 github.com/consensys/gnark/std/fiat-shamir
❌ TestFiatShamir 90ms

❌ TestFiatShamir/bls12_377 10ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 4508440044023296702059250297535132491355957263901127048922215532174200496378 == 67269173636198734863304843732924989070129095650689098566934808116168714563
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["71317368782694201358166673745625277806434085750743685747776114534330033369","6835310039567818198106525249593947736962461932796459677889035482066573563741","6144425965600384008965597170634497284503667870072465532869640660784415432719","4899501654645036946266275539219213275602341044858715369781500208936875199618"],["898180772429468403494951378633994176656610709485477194804959697207602205520","3926418948279465998959731456530667001514407610129715719368129518928499936098","2315947057557844675398505412870533673941860453254065067310848178078810917938","7577361916292008091904552048324444008951120372486617684817219507635758033366"],["7342954118491511362568424444362285891885621379175947567808721742168532617020","5148808710032875629985572916084231397508848886613774822644262426133917132086","6908800688199973820678146839007233619458575769527096824286069435914956465057","2407343309663249505802992225075455159320544953970299962152835113504084828094"]],"Challenges":["67269173636198
734863304843732924989070129095650689098566934808116168714563","6362137256370898872014908262472030263950807814486575504352506588456715239413","7107976163818349630755198111985836262387872635424869138149039283717261116292"]}
        	Test:       	TestFiatShamir/bls12_377

❌ TestFiatShamir/bls12_381 10ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 37722347424567892764821629033587800791142731750802784316138392473217695384677 == 4690793243414798945845771969929228998076979340848000585500428315964987683745
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["3703401716390331867583919287384440589857095776455052598715178883122769681205","18964941565599305699176758973398217855076066727971775240936741579636506370449","43071131508484089861085888698494624205857689611292132398917794561456595132010","47032231232727853119903865478440610784846188110398949351895551325333902165488"],["50255968274404749152780345500891540183184133139033606918085304851015590722935","26389339973107715539380682098345994653414740841029364938445764717238373970591","34523622262367528643629876656983393326466550138602475433178113960306254020342","870877222281880066118257332077470033293507633568267463530012621309437581956"],["45232733639773866068265845068577648641798099092060131363136053854403893649006","46665022440260710878627268694689840388689770987831980017703670583517013056868","20904086422134350103758814839161966615411460435094766812709896683815914517185","19454189379914446001156918405767290365231097810928007669679106350930167309723"]],"Challenges":["46
90793243414798945845771969929228998076979340848000585500428315964987683745","36823437827841306812318763477947180460612659429579845079900877104463870694162","3784882553381520340670560358537844469972361895339627677573135015931580257749"]}
        	Test:       	TestFiatShamir/bls12_381

❌ TestFiatShamir/bls24_315 20ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 5957155044198812863437558969315670895377354113648195743615213218173528933369 == 5794359339131945893740680880305752924057191498617471018884177285399699169055
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["9235911830608056487048995996978011746660693389925977437370597387501080819287","10452768857744210518561322089038259967634255509080255726480864355410367517258","4596669718155831685949895313671224936956202061020528921042895194358902032083","2348584765200836731560526304801573652430703636014833728335305319700937560668"],["2955700112332182329316595130229750282184389153930936379061311476300891949011","1934797441136601769667032632776993277011802120802268091672623732364324527508","1974719270879311456589181313673212258530785668191422502766792051828582982819","4118652008210825518252251133165945828102573056389708635234602246078310927124"],["4617019813868244700102812040019990246494134197639313187543232148613414959810","9289241643090472921174130496093498479058437364332111509072209851100678790975","2718043602230793040944520535893943075191059300549663728915985825345436400546","6961055190821090417202838033005302725728265036134437144112754374704146017943"]],"Challenges":["5794359339
131945893740680880305752924057191498617471018884177285399699169055","4951126652169683164631753791053361851796078719700586985243601277210418335172","3182904260098275930901687080012199840693864848960097170113316182386557166741"]}
        	Test:       	TestFiatShamir/bls24_315

❌ TestFiatShamir/bls24_317 10ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 19688785327372956346362112105906083129787030116830198531910118266211791442831 == 23327175990519417498158289202479922324444085729575259160011625687611249490462
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["6538421556382004468658190218595468623900245533439382181635487894671105670880","29456510685481848757249253024426136303322192087873927767138377180150996861947","27949825721761095141278543000868736287142459249538971202619866099268005846818","25741821796562020910082260827825397916184819322406565895103025534532756597710"],["22509553395137099848061592214553905555730684396793881778277440253580904700872","18952254400133448690781357086591317260776830175078479054428011520685728045025","11786535218634317051043563851172872645178764925851035445326010964502146642379","27445029917847247391611791913324153370049842745685249520724547691360516942805"],["22672860785166186723580062115063051188583797976217730591892145025749825412584","5791902756660776502062233111311048613039815773689370644189809069663790780763","25621401963526400480057441035471461819228613755743415993594371012679554429273","25588234378702786539036726324990549941692986608479122576570969067269142873328"]],"Challenges":["2
3327175990519417498158289202479922324444085729575259160011625687611249490462","3343496194271944733175238493166705296428846148513103218248311149992664045173","4304602248981853851118971140062933300395007327810825761859294473232372980066"]}
        	Test:       	TestFiatShamir/bls24_317

❌ TestFiatShamir/bn254 20ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 14584659515628104654715122704318870873994385298207792611179682798019862421467 == 13363548754690716534931062830448948497746386595954289399020963010086767537110
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["19277327168386949952854337996945075277267950624058683443270490339822361216202","9172995984691329893709465223230646888594021555456936757971091815804815865493","8006082898579088635179780134723293340611578356140985318756252143576558805002","14152203719881346366569130506000492932912895813108411166358408653314719083965"],["19576998862520901388388126745693230892134486353253742865770793192771659026183","7553783328051800958858875895096512317978585959704556057086416027184462921852","14013284808939059981768405297989870306029058158012104379160351572281047069696","16720974654196458949699035859144502220506563856774690870581907848880608493360"],["12307391707169383174094162745320739277122162811073489840039355573589931850130","16597352803077553145861735747217764089606783452457094572601386102417016239817","11267268373472291225267916146225706086545644074540731009775177364328708198244","10755540744902237826028924663796783848569833713443828732841615061762283419902"]],"Challenges":["13
363548754690716534931062830448948497746386595954289399020963010086767537110","8301388299594191134741379825478955952994517590935971938707903628158670073258","15184760933563981753584610612629914367810292971048348295753118001221087249412"]}
        	Test:       	TestFiatShamir/bn254

❌ TestFiatShamir/bw6_633 10ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 407836419104595395564198769664595077881178962744170365315810434390974443880141879268032053156 == 1196753196874382766747999250375277162616651243666462144424434470883105985186878272846619652299
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["36629208289834153876927260955453799589655107599550895623569294544094793106821121031262831076093","17581932454772050207210400474971470373027543841891960931467657723545327571248055456565598091805","8911985451294612175675025207363701433064963674381717681934137322530613457227664633096618110271","5537012768990994495173273190982638052458023579842517603789166628917997072383229979731664498711"],["694386878773439986022413572095481967986542418463619521013900889805758115535118499649272892073","30169070791031553280228748760532830906600846405631042137428714785400430778145855478449165637032","4842417605315496012835180277719518184576304977554165375977689963862383258383098743506320289116","31103121873893777948158385098682347394084146337578915809895224366699117432606916269039513519439"],["32575847251147488672621455273147940894417654594536537920931377402087335876362859192167619984122","324185862574345810181146318925592558863277956818622243641653332885209183774937912854464160017","24
958930649832944512408052355943100350627321066290697636942037664806382607238814118039064300671","8089974310826900732085143754753760973934967654708108169975653530132476449084910290884294097654"]],"Challenges":["1196753196874382766747999250375277162616651243666462144424434470883105985186878272846619652299","3820019090751576969137792809550791449088181966160098448562343336382330819545979900141279661002","12237454079847937321133157972705911759094475085153901218718467885422331032774009280851108631631"]}
        	Test:       	TestFiatShamir/bw6_633

❌ TestFiatShamir/bw6_761 10ms

    assert.go:200: 
        	Error Trace:	/home/runner/work/gnark/gnark/test/assert.go:200
        	            				/home/runner/work/gnark/gnark/test/assert_checkcircuit.go:50
        	            				/home/runner/work/gnark/gnark/test/assert.go:66
        	Error:      	[assertIsEqual] 159525624352155736782079896064815240648472512686716959798758664210235618581890149032363863853837530334067748656152 == 197495184541091399967454482577368934129482594302396734094107817085815083980183440863649744545117570909726923458530
        	            	fiat-shamir.(*FiatShamirCircuit).Define
        	            		transcript_test.go:79
        	            	
        	            	witness:{"Bindings":[["254701420297002989443109878670141886491443164685742917079804231304400070261875930578760358995612385707161702187064","191958940337025727410327187452760289610150408689801437972347397385987948202978939053605519595362978613090901400490","65369445550291331191931413879322608279715080035158514950836113532237303997040557468575781547397112141609018180763","79791851534160097141701034256795061618914471547086084173154007173242214546440822911173095847457432512911366709606"],["26267752777403986888738739047846135278191131172749667778776968468019385482724306214342276174026241272207165648051","103992176472296163491681234581734117575070499057949247878376381019517239201568230111860676374976043268594992766648","162713894864486516304318143882250172295253999063145201699691415142926419974674950313604253323120998367475456160047","154455795025248114671743019046711613018202155658277825368544221204906032369683372611411487575890655172882382468743"],["142183323993664393807207534124687287899224
789506682334463447867367537078675916270187216076586228119986741730629862","84590365146456780933668514766429643755517934094530519489965717120816332540045207591379005487295343322658552711388","171985944051342610295382629234170885357850966017823406309728133941696535061653751938511234314369969687070902819222","232475017834867131803016506166645174195925638485617294023314879806935712951007524680843451226666143129875269144333"]],"Challenges":["197495184541091399967454482577368934129482594302396734094107817085815083980183440863649744545117570909726923458530","8753962019236583997188444941781044039363521915445243486113541975816297230564060575479117596099127731462840231001","130661421406449811740947322576983721250794753070747235974752021528408038028762732393478160067073739502907253047907"]}
        	Test:       	TestFiatShamir/bw6_761

[github-actions]

📦 github.com/consensys/gnark/std/commitments/fri
❌ TestFriVerification 480ms

    fri_test.go:128: [assertIsEqual] 14888521443990119378566971289455002049989954170276743704794153136116040821010 == 19366473084011384333754425266347258028345720879945468408490361521685954498047
        merkle.(*MerkleProof).VerifyProof
        	verify.go:106
        fri.RadixTwoFri.verifyProofOfProximitySingleRound
        	fri.go:181
        fri.RadixTwoFri.VerifyProofOfProximity
        	fri.go:231
        fri.(*ProofOfProximityTest).Define
        	fri_test.go:40
        

📦 github.com/consensys/gnark/std/gkr
❌ TestMiMCFullDepthNoDepSolve 0s

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb7688a]

goroutine 148362 [running]:
github.com/consensys/gnark-crypto/ecc/bn254/fr/iop.(*Polynomial).ToCanonical(0x0, 0xc000228a80?, {0xc000364760?, 0xc0?, 0xfc6ff0?})
	/home/runner/go/pkg/mod/github.com/consensys/gnark-crypto@v0.12.2-0.20231023220848-538dff926c15/ecc/bn254/fr/iop/polynomial.go:338 +0x2a
github.com/consensys/gnark/backend/plonk/bn254.(*instance).computeNumerator.func6(0x0)
	/home/runner/work/gnark/gnark/backend/plonk/bn254/prove.go:996 +0xc5
github.com/consensys/gnark/backend/plonk/bn254.batchApply.func1(0xc0002660a0?)
	/home/runner/work/gnark/gnark/backend/plonk/bn254/prove.go:1092 +0x31
created by github.com/consensys/gnark/backend/plonk/bn254.batchApply in goroutine 147437
	/home/runner/work/gnark/gnark/backend/plonk/bn254/prove.go:1091 +0x12b

[ivokub]

Hmm, I think I'm still missing correct marshalling of zeros for emulated targets. Will add.