Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better number generation strategy #735

Merged
merged 13 commits into from
Feb 15, 2022

Conversation

alexeuler
Copy link
Contributor

Motivation

Current uint generation strategy doesn't give good number coverage and doesn't catch even simple edge cases. This is discussed in #387

Solution

Create a new strategy for uint. The strategy combines 3 different strategies, each assigned a specific weight:

  1. Generate purely random value in a range. This will first choose bit size uniformly (up bitsparam). Then generate a value for this bit size.
  2. Generate a random value around the edges (+/- 3 around 0 and max possible value)
  3. Generate a value from a predefined fixtures set (this is helpful for taking the values from evm executuon state)

Results

The new strategy was tested on the following contract:

pragma solidity ^0.8.0;

import "./DsTest.sol";

contract TestNumbers is DSTest {
    function testPositive(uint256) public {
        assertTrue(true);
    }

    function testNegativeHalf(uint256 val) public {
        assertTrue(val < 2 ** 128 - 1);
    }

    function testNegative0(uint256 val) public {
        assertTrue(val != 0);
    }

    function testNegative2(uint256 val) public {
        assertTrue(val != 2);
    }

    function testNegative2Max(uint256 val) public {
        assertTrue(val != type(uint256).max - 2);
    }

    function testNegativeMax(uint256 val) public {
        assertTrue(val != type(uint256).max);
    }

    function testEquality(uint256 x, uint256 y) public {
        uint256 xy;

        unchecked {
            xy = x * y;
        }

        if ((x != 0 && xy / x != y)) return;

        assertEq(((xy - 1) / 1e18) + 1, (xy - 1) / (1e18 + 1));
    }
}

The results for test cases that should fail:

Actual simplest failing case New Strategy: Simplest failing case Old Strategy: Simplest failing case
testNegativeHalf 2 ** 128 - 1 2 ** 128 - 1 2 ** 128
testNegative0 0 0 Doesn't work
testNegative2 2 2 Doesn't work
testNegative2Max 2 ** 256 - 3 2 ** 256 - 3 Doesn't work
testNegativeMax 2 ** 256 - 1 2 ** 256 - 1 Doesn't work
testEquality 0, 3 Doesn't work

Also for testPositive the random test cases seems to have much better coverage:

New strategy

/// Edge cases
[0,
 0,
 0,
 0,
 0,
 0,
 1,
 2,
 2,
 2,
 2,
 2,
 2,
 2,
 2,
 3,
 3,
 14,
/// Fixtures
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 42,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
 43,
/// Randoms
 362,
 516,
 1120,
 1983,
 2760,
 3585,
 3823,
 10523,
 43672,
 238225,
 3626729,
 224284888,
 494423685,
 971025881,
 2888328457,
 19519174016,
 30184947189,
 33396775196,
 56821726909,
 689351498871,
 1190816741152,
 1572831182273,
 1591727194015,
 3826546332976,
 249543156942240,
 798311978216765,
 3373273342330838,
 10918911084916060,
 50189871435753677,
 439725383077241022,
 3401696951181245404,
 157496432936802028161,
 2475231153114891158999,
 105909393431246995593069996,
 107913729131805239501602104,
 24631520462805559999770369544,
 264575694087087137389232101050,
 772619611430724619639690567891,
 25810618386653091334149509441325,
 35915034238800349492931614605635,
 153506003836326203094346956829482,
 280873127667185661829222071797230,
 348010450494708774993655163931128,
 7646506327056416587716103348256532,
 15464545752295587733889458682609054,
 36783397758511900616387809774631785,
 97242441411817996925069849266888622,
 147024006770563177609312099458628927,
 188121887855574525305144619055921663,
 294267473426187474747849162104267339,
 12724229308417247999730640201484862607,
 41014563812140591401469939715597749206,
 156238440400277946256505430414263332398,
 166765039863853147931449965207370754419,
 1612415668463703462047366750421648918542,
 9908927435109728333625167925598760060978,
 10236965621329184275422787688640537603424,
 31494523084140425955552978549740706561713,
 145896957048859625465331260947454550686895,
 2238492176002143802154295483195393306607489,
 2660533239178328320330692574533173796089207,
 3875298189000802595481276526739996399445352,
 41253755041548665158665352072168402956844604,
 422943545645126455580859345909514636244314873,
 4999417872146633167473658301209717350299941294,
 42678878196096467200534716644006140216153080986,
 73553348673874374231363032938940863498528392656,
 483784367006201018773058818081161556557759349649,
 774808749789481306668324116495307910856771895702,
 908847594477485735591396633334039779589105452351,
 1262182116835146393282622336113040120220695823557,
 3329270524375098468910898521360960753226504223783,
 3333194114900161925274498029582379652285031451877,
 4484957434972620090665048730245031768309952819964,
 5145409819232473991434653933523417039570520313943,
 19464760408319993244613164353807065092392295101253,
 94545897941650761447024922888213559969940232943899,
 337470395661202575662318275605509436970557341018103,
 585707160440833113127889378837557262626096740931953,
 24675058081760600058539159792343952141141279524037099,
 24693393888541712700369296852522042203731352049360635,
 43059652374527114156027031453657075997130860134060441,
 269545940812941710519360602854087382977217091144974454,
 2736173204671970310280131504797900563536755779315861147,
 7585906577758645185939204117101805431316119795885588896,
 33643705350635296300819922614197587563522030511778036139,
 40032055533762922159402968666283302464274464144923691558,
 105329440441726119029792299829277042856522968630327268931,
 389507037207441101784969622969498565152282121345399912898,
 9197023485487217168386170149318496992322357660912816895928,
 63801834251340751471942147909651277976280476380096593343498,
 89759368211746996493899901894807549508400975163659912560469,
 11564042667181017501113110131615327390144857528086108829035938,
 153517312332071445578332558429419487433099243064802539820541823,
 498549209783301696727706981293591805749770224824890247634359953,
 756811607379857002698587296372743329408009190047460551482856459,
 1752098029480478965257477307012843483667278095656875086266774124,
 3392734207135717592412363460475926260240793532828134423027162734,
 9233700333011670390951211460577308474680443838961552150737192105,
 177833816274340328629934599844888363890654985691221246047768165741,
 304945838058022981508612894265691082275151317069149827616755125668,
 1192324827720452324949310219536112745636227566373302366565379083793,
 1553833874442004385184794115206805717524058675081061332727524907963,
 5042197305867743650008384211331080110564829168804442539974363464481,
 23028419957613895322950954273368266023629068017559103557745414248086,
 24161758178766814440807498232474608720723424436663115147226324786949,
 71146866016584996348526549371428773883039228160645422744616057724977,
 393029566373024173188227121833346144672826890067408893880564491145150,
 774674628535569613958038000994363892071296911046562735013643101784688,
 12613298676762947553352770285147304609152249108003791881467615352343351,
 26656048534039732624971028050044574640844635346076931983212285275712136,
 68422598250788211161165661498169242292988820156910420498009277042385031,
 256617890012953052738992627600607351153778429589727924310472202100958499,
 616970969673537699119375656717930760404807959003040730274126076713377039,
 1555708305164875762757615680564475585448617155478848596221908856216367409,
 4054903716437203972896858030663045427440725614625622617583124788733779570,
 6865905988385269572457253830501532729958116142363164033207340240576944376,
 12149691792560093164802418591014018621430198390474718925395548613494311534,
 40075592997550927721721307224428429558167630595113397027314467597041717692,
 97645443615448263228891435376312407256825228331928581250064254424864972300,
 334700497169719979843538055849453709550670251808812700026283322615592273541,
 377324214000696227091318659128408814806908706594473878876703907831739905599,
 653118130249510828764236419959345954288971153320462048408042919236571719221,
 880545739739058157244327127571071199972625550212799644334011478745870822392,
 1720832386996471385647959087994963156388682097580943821771954323744187554529,
 6452590156306434120454252602081937759544560305921174907743631493649951934141,
 21100963070613437129259458410956980990254921083177423324612011912088642821609,
 74426764287200419770528502184548186513942454060601551250796841729950058290695,
/// Edge cases max
 115792089237316195423570985008687907853269984665640564039457584007913129639932,
 115792089237316195423570985008687907853269984665640564039457584007913129639932,
 115792089237316195423570985008687907853269984665640564039457584007913129639933,
 115792089237316195423570985008687907853269984665640564039457584007913129639933,
 115792089237316195423570985008687907853269984665640564039457584007913129639933,
 115792089237316195423570985008687907853269984665640564039457584007913129639934,
 115792089237316195423570985008687907853269984665640564039457584007913129639934,
 115792089237316195423570985008687907853269984665640564039457584007913129639934,
 115792089237316195423570985008687907853269984665640564039457584007913129639934,
 115792089237316195423570985008687907853269984665640564039457584007913129639934,
 115792089237316195423570985008687907853269984665640564039457584007913129639935]

Vs for the old strategy:

[281368335871742058896344557338774872366766563217730231841881112605204297366,
 364822410079358517098224268071440248607348375692794793607380941113804777055,
 672546918269513596558989341105586824601417954154907112838697843942096072115,
 1058929417546455556233246085153574268617733563187603062185661582575200539884,
 1092250115363355029315423949983306203573813825245258733460692885814877920271,
 1148905770612565908942764344282703230821022417434948496101505120498027214698,
 1376812423925057181312682392286932517627608730990023207779301907339976433379,
 1492899683917851015748536704907582198298836984276999825727945053776533114317,
 1808744199665193641997057689707699216368520760052478956910956391197884522498,
 1958571702806888710012063123272840502862701217727393915196715353129947424076,
 2168993155629317088767724547657683354740032280211539000333066091983655670220,
 2763810300708239364220867386436066356474002186827096815495645107788964613207,
 3171591855418114543672984720824053335863495294407669936378497414555192490403,
 3281825919184007036803240420677646271649226306898480590161297221488479867585,
 3497590771933655892447939780122246331325116506860788485689038943171387636730,
 3826557156563115154683389372655937009655926939896027313824758732229923129451,
 4579516077608105408074339211434219170213662817212727535297377979600271632483,
 5446246782069213914253651610944338721802079918098320260719148715491608195214,
 5932969737937012243948273432522126579428716324596744281174506607127962488796,
 6311544616781286155276055527404781175064692735094515074196466338249915353149,
 6346203270542184858612425604930144102953958279911073206692806930554095634477,
 7899493556793752463222774499428460263452242096966653024114659269843492662490,
 8364790083911973872997656080820739544103725010120251021098742162223448126932,
 8750554909505181819817218930638035107066924889851540670200955925024256114010,
 9086806469735868838405155643054748929699018152314809329414034540279634447499,
 9159697622457886374861291477595908888655037434363816795334466611529934801194,
 9339173380425268301537757122582091850617639717748761800340416258367792311584,
 9498929518889314275512552652015129502824272294103307011837995724828047074263,
 10054799463480154843451883979490661185533100899721811398173740004023198333030,
 10408296871976604144918398094836227651526897410847601954730860995981171022862,
 10961164945340778977801392597259346498517256164748439439577835289550779305813,
 11518558759946908659554486496789172302980807213240614639144156799665805937192,
 11910656429264259647871356954919829837863369769412803195611650580161893436013,
 13150669723117534823866186514463756062383308257168157647769415591364761415869,
 13171927252148089006203868394680486434898735272468936811825858351483134948839,
 13652188683774563767623080422566939814967762017628319693408149886623951395351,
 13814864382323242911587132995070213956235512088028878887082521350669188445247,
 14182913814438305004677105116227046188391210899465524585858436203736789079401,
 14366199463224216598115465076397019374751081387237754021736755520500830327346,
 14423805840527037366275208846593509023820989939587385101858172830472045743399,
 15066893141453916759877882789958913776561147787935500156492509552224052251908,
 15391339551013475490557537292486825967382286728292267385064063110839585099287,
 15573986715396684706724423195611452505565172190124414493596296339175719289479,
 17677833654453513649000589822401808493196529390758876902680995846177435132814,
 18561862668387951005712882746995009019051260224489390064167195522246959640033,
 20021142367638015534160085305782504862845403233425126235255270055335088131984,
 20640301186947415938931578514758320065673703036454077651642548929382155683179,
 22319222890268341725435718561229697005870901031270292288387133677957650596589,
 23148493171636193021919936193151329680921981756965063608661003798887994564398,
 23184817459418149171968126449934370081208489693257798468115622065654614629563,
 23258435276857359381023641840909293908708502909665213251493352529768608982740,
 23260652949912882760782624414990222803960339140485239198683426350143602394677,
 23638099691346198176781533210386824075347115460207153706373234072928138925050,
 24426595524371211553628209027704712678799307826034427494723333556322184911961,
 24456630412345574391364687613548312056241373990338181971711417943886021095475,
 24746674332636135035269121052115974104068539413085015800280975988930162113401,
 25473315909537154447347952375895068692152001155486908453803041525321878223502,
 25911347865920576933842524420969391057340961522418305248476800394862660898934,
 25967765598957708945053105493271065963653529644327477537952740848446180541821,
 26832607465252822698265411110618675353029196647070667888758527550689749616729,
 27123572818448044498750896353580044717803483726032637665339461658913131406327,
 27247349778804082929025667032726185610311397580967808242668270516348999292619,
 27384565287182859703554086950836729588202814647841123081000148148267895519636,
 28340801795482741519762594709555123222933815562017994915028753492214825266691,
 29219914066073856605280322949654296453558511499864920642250344361526584336255,
 29265359622247104137979639142120501399068986141661920956412572211849680726802,
 29280594046728736609553405162773033307837589007063879321962164427722120036529,
 29997467047628624177308442824569385044050322175814184679907927154369711803723,
 30682222906125259400580255231366557944781731619425434138826715282632698043901,
 30852676781839119478675045080875923164525770388942767107622237666350213548354,
 31107972695526942525845926988246757406299476616560838149016617284976488403058,
 31172250332479584397895764366902431254599373326425828311160893255201046816444,
 31572956566523306186739943761422136355097061233698338004189552916988769582215,
 31587843994979503161964491457693441735991090405991509301502480698102039104999,
 32371606920989315333193431161614716202395129138327031565923220565145639863087,
 32407199309510237715450533004931429575352067036319705499325024626405254556166,
 33391131036069379263899471562894096489681848038064341567533912477083730642594,
 33865610434739098519788111402876908403934247179107768378257987360926051162120,
 33919552059474942281867920029540739260409681073522965784130136177487513645226,
 36447458907709573215068948724910956667308337810956797647261530206963539242540,
 36828163334778324604040323618534731799457845174497580586637690739647257755093,
 37500014065845627368479849098142753330418564357417717226433659854155805422614,
 37717108238667492519912787149320665314049376481189154978152569139132487578226,
 38343782611930866679454287707216079102949610076538935560712964921861227547106,
 40133864809537318716398559806056359612569253181449779008498225936076828228570,
 40461502892324216640568951782834701212866852047502260797745361381187677117655,
 40848436254656396609957027956780267397496811781252100233587741401305632143483,
 40909048946111117210354890813344354154191016557520019824834713739472053998344,
 41234356887539472631042490865834561172069217543778496826320568533045643430710,
 41366028451975118576983213813356514495315864549044508634374191231727914181786,
 41670743014812810180158456856101328679532407396450713457006497321824120659296,
 42564044510848505653485151886618641202870762031152586496754855686937751781422,
 43037105919295586364743208644659919867532947542634374031297371584598591856094,
 43667440895023489023430889436581291491855448456600882233941949498991383954455,
 43875452099997004626337974571919684899459793565828017349870376822162976894854,
 44719995685022428714498512789462302782289279795051331963838153491982859994304,
 44992013321853608507767364268740046524058719860490975738024150882905742142068,
 45347376920371186265056578423508915467555621102356793577860264427609407320002,
 45417822380118456831343979425557987660709928309474456904946981499806740396971,
 45478408061756050208496518724135215946889815335458832442019865162826498078437,
 46094980464553200567008070802184840909322819924896454277033625251648459890753,
 46295949845529146668170374774611529635778316518433103833222053651638153856897,
 46399689838288259600628020133851246400958140826483446503630589452655878595035,
 46711886120057665889623299434728495286625874764116366163309546867374528861763,
 46926565577013103105537460930029566918699334505330104074818487711313324097131,
 47278772767803623640845662508732973685392711040019117300421638029919043565618,
 48353085275969736118048487547913561929051162321892458343999326730917957898450,
 48363523049527155434459282024388309550645884431099550243581203966302339951336,
 49251563797255647434922748070452074911558066792600747686357384487789450970811,
 49687168906137435292468753452059026751507497238115674265532683378873998363487,
 49920908898504662176130221118913463912912102648275335521219172719596705249271,
 50023447766341896877031009951417549739658142617664140173419650797052992787426,
 50253915704559469075333913375174021370656552554758775643338224384749856080708,
 51401638583009542604341455581875414790423510963641977437444116374969891474779,
 51541388322999882759664219105543638858909326752975905549423637803633062611752,
 52176648670763064440128441150800723901533395974492373735992979350635946700222,
 52559288460945992646097885350251797612855950672730954352957393652879968697289,
 52623248117731165827906748766933632485255304158668335086974989354181048554026,
 53514809494841140721438435905175317731585602039275043947691987918954775374632,
 53549794021079005853881967830715814399617769368511095774847130915162423752790,
 54147749900070236169477058358706644922847142700631267648938502431596203298423,
 54395208081560387608117026487477971649894080404788929134351405426075934329946,
 55329442926816137564184894370399919769815693306889706923904995951196179497492,
 55408485143148059883977982727867873498544464493136290553875945203203695738144,
 55733884524172574834232549721839111797173585850485800511973047202091240202566,
 55840968040995954149756228089557266356556796383225939145099973963274347384352,
 55926550792763397635029922283523203510503331040005186535160542819914270173126,
 56001007928207461315200157239941013754083206501912799760697932679617475173605,
 56050650699325512512745979656968984934471417079856047371429572613360414227457,
 56705481209762160125342070521537449608790562711931140868425777631273297635908,
 56791945938149165047201462207111427928170047706806002376670461498173485429085,
 57078109831864881886648952673093143218446719956618345737550133770599146345749,
 57765234955930824012310843699108277714815262019268146763065570068254637170489,
 57971945068726193892914197984837906360933770626389430685795948508264446508634,
 58106811824607544703168606370250709780944642572525938719437174999073428062114,
 58468978837362248602543589460355029076550321946760658505214621284116034592101,
 58595942900830379745791841521355566870018373458204332517883606855244032877362,
 58898813025042386868046168427667280938270989093799179415822883039327602986831,
 59115934570998766522960989336982127576850774533895414941586732304770837093490,
 59205142184624975919813989320931010132497308578757131303135055800252577319495,
 59402289718140102026696083458881583083601357132321064122935543161867491533399,
 59788844535053828582526886233589101174123790093821313922040539383121575769964,
 61241220657296481963987511147802527744507565979767943038243799038291961260067,
 61376117755531490395469922857165018237595133005647170251353517410942366403084,
 62497057779849511396357936211175872748227965785972493355105013474445112904693,
 63410223432961024123693355247131879890709941397665760887809738960578147213996,
 63481396021639143876870669067375103747010577754451633112092698070710916729001,
 63540550250203759534279085979944410432955065380525677064867549439265167957772,
 64387976884960271133678939798531109814133584620900264291421244956331137562202,
 64396182153890293361100494505412530755861548712382154168472146474993777703347,
 65700213911183366252489233279747188757669392957280559760683286895485414706924,
 65749983635513007927476219354184837543476553051594746887976865563646535903299,
 66932201447014587171637858673256768136303884748947271342200140532923710939430,
 67836234141422329443355645018810899385598035908506653428141494008602284234569,
 68177766473428543220462066137063703836098901696320096880474659496805073224345,
 68835853785260731752094792181357388116692317906402126410756270182108794324016,
 69199348245278747238705539679111383885188506359532395987889709098936830343186,
 69466120600124453633970735662743833643915317682914381577162326698708238883214,
 69516833814315878016279112644897247929419585497368168076226285953729129302934,
 69666627038391786702573497008089333583492177902472096578185043067483555535958,
 69721750503682175455102300033824816734792527213672901733484457649410835018544,
 70984538655524851536873030324641025375069380206087520321582195855106221131841,
 71308514377494774200815969057965407488582388463387265541174007060563485801668,
 71413017358236813940608057660346964402322821338639899158838486488833554369504,
 72498584686120639569731540481701299677003914656851269023018652891779449130914,
 72833955355776620359206858751422615204207086244504067153863404119615324367547,
 73587602765563698351834070968203311936963016790233344690076540511863183758845,
 73796316103423577749467093192788339760207237704133514283606545250425157166365,
 74044999892771930014627304989839762643740252260323181050685530818551309874546,
 75181440209015207594867248174849136503866630239973857022938893211004725253941,
 75618242050687752936784184743304305179973944229765130501418797528240204514978,
 75796287120298594848492563954347271233614181683058112360909049621359291082349,
 75869472459279364550893826726516412773990118860104345357474167036436124574157,
 75934153389671783611045582347264666188849899651467751677556157036157580452079,
 76373451339745884979396291337791765061973321479154364076090375011458537428756,
 76968139024168575231997009295741594726537129099534887831892525082792777414674,
 77322026317803766040737014093189102578890868491321854121717405701157122427706,
 77367772403256910609634962197516574578307178701596651562582888728182011882059,
 77392589460327003676060517445510808653395058278054683122348890474609946038831,
 78248831458852949186214946123982623825022317162415473370370425870515197925912,
 78296397814033963204190708388943777290974989893846217895750576244718301551699,
 78334487344651341147640459289593130307998267522210434868005487904137441978192,
 78703257990038361968211091104063772642358973099761005790649018371349002714498,
 78888433166381607324428234509197484690368813924947034221644783223691038514464,
 79373163167781564036011819412919428859141183094579583611575657744956118314024,
 79616126515542782530102134687461906323741662342787020297003153441915449674636,
 79991758855211779796083008881727281331516128554919640711334830335648574050416,
 80243206676689129966358076868086229273791635098018415384889046637533398322788,
 82120632428193722988055612836328498219767053246853481151983403226961830394205,
 82233920043869417164711949585035964987594514950981617526875672939857167764178,
 82292367206668342894451673099730075661987349346785689274939996270031874739309,
 82431752539139532731853498349319680319723111338094199542709771444975640506771,
 83377414542553838706442144631646207682265925130924519635823048725997969357450,
 83998725954774903556665907805958231804547036540920719794559280986217844934803,
 84341457558471436433156138924530570251451002990301126091075808520263717258832,
 84400129906591804641883478070046054387994696878246860899568686863688780704657,
 84574525440665148841492350943503080890676902717454133475322802604995167971174,
 84860758905641928413412486685599376781355253725342240780646558564499719432612,
 85282219530679891155344753543127825873957290234623852461031957765824216151281,
 85724769482598696373881352730885847865173725250700393924496117066610189845048,
 85726276628050789940000478368403934526972672496283156461225662264269608740838,
 85787180838689888566501637961738142209723208068289801082555890181640635827662,
 86612210792178704499468933192817326364533953150834726528661494116722073958934,
 87285318305246556374757922300415160826405735403995785438506197072294368026268,
 87386768790674747433716687855616113658055696194779879148554289168097947090299,
 88025070060980925793243748267897039909996436346382181522450525071990074404040,
 88367009274470909936572331308517786118144676623665488156507776574061744019165,
 89728630392426232858188531220994710744483993610296664453773801377152515976954,
 89982212567700009659895046738451799742672425537059302434767395705164296215641,
 90031907766883517767734898911304516950395587984421334575213842466969307275225,
 91089427525723750397647900889967255016399575154684493042936332654083369428484,
 91272477829614932833515778196183905278747886922959863425659845077418597186364,
 91871247436504622060373600958241935939715595246389805636773276246148899797387,
 92640246314237385318414377620862007063053566623598257764965446889404397727816,
 94290682436874300511373745265252443057335627820488064181798816130180251878053,
 94695198466289409340201378870405941691181153180959263397094805622676222712769,
 94935571934040236363373478373924979102334436125388906957491372656967320157790,
 94948797652067878987324413612072342543513834357767474872535953704566251819324,
 95067466782613841008065265859516544777031703256279791120541922363703450338475,
 95147147647358394706512317223027480866822719801061341095141330823228640033033,
 95153534177699394936529050610775507308758021422633648919495870878335026277380,
 95401530579948490153386151339673631423474039163941250574200338943450894355194,
 95547396502086033391543016234884195047609360100426380907335666603494987266253,
 95549879470805274590844686065648977016360052340110570397093426271587430510891,
 95749616772509372330713511069364686910838503772574633071043853144786772036874,
 95965924516828152379405204465123705349729611268492915383219187116091853540750,
 96115997014063864206190847413929788639125274529414006017020950060605348015883,
 96443954438835472424775809418461506408560371857946966446141187532196075380936,
 97067087735182395481789219904626968140703317363633150771298468737630911176075,
 98789761448923708213475376454404759210848892801306665737776275829009782386455,
 99990172591882036837232318907956858819939111526616928764181273771382588767616,
 100038236203759853907038541444107944656411284313170143034616369234578605720067,
 100098000386358391689866589073454084042392816485441455246023611299894269884670,
 100835964828212917864607589643403901184514742654095508928675168693900783233489,
 101635148868248982548744397187417395747459718126269023972708711914784778534675,
 103100070202241256223758522138512462872564669441584396105374238543530963438735,
 103333282743348132089012849271820950514910162404814383513189875696794588178502,
 104639846240150219180036535750049322508596520901153003723321078446616411080982,
 106529164764628082481372443334365219327065486905931087202595205195006212953281,
 106678753497672178557474806228955338496381618393076431944178758389892363954499,
 106732799077324962917392148857044407662856140613488197938939243217785947550390,
 108136498777453412629280938822770974252559192560381094010180888749244919624770,
 108136624620358440358806024489849716154890329832124825917191641717152373848426,
 108314034206404123139993834400943802684069567153121749530821874428129007190425,
 109107636082678171244804433194957487414824736241654653448053138275034371566873,
 109579464632464298438252448897937626937715592812004399243154516333891889428109,
 109945754164846022653852342067238628416213796289107851811292555431664581576397,
 109968236822119436539487877062334545151614006821631639677522885013058077747947,
 110731006973209067190534064489140950228367403607953993001689170435604483179742,
 110815158410020991722490728126487881657832469665600400555408902832348509126911,
 112152618170993964143484270898038639410772952738619333806176094496508292900780,
 113105306412525046003188083087697496545890179110165935741204785540945407386800,
 114066287605022763885986821254799874878690795159108600632780497321773313488114,
 114278584475928120362089794977603356304621039748568729596907017059439538973126,
 115346152156686977625121718652115706907111956957881716922442193647684774696360,
 115498660380972491216195481987492563684840686543789059856440455662609048842435]

@onbjerg onbjerg added the T-feature Type: feature label Feb 14, 2022
Copy link
Member

@gakonst gakonst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it - nice work. @brockelmore said he'd do any follow-up refactors on this.

re: Fixtures, one nice follow-up could be to find the constants in the code and populate the fixtures with it, e.g. how it's done in Echidna by tapping on the output of solc --ast-compact-json https://github.com/crytic/echidna/blob/06cd34b2e8527028e64f0afdd353644976a5b97c/lib/Echidna/Solidity.hs#L154-L185

evm-adapters/src/fuzz/strategies.rs Outdated Show resolved Hide resolved
alexeuler and others added 2 commits February 15, 2022 07:33
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
@gakonst
Copy link
Member

gakonst commented Feb 15, 2022

We seem to have hit a Shrinking issue?



---- runner::tests::sputnik::test_fuzz_shrinking stdout ----
thread 'runner::tests::sputnik::test_fuzz_shrinking' panicked at 'Integer overflow when casting to u64', /home/runner/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/primitive-types-0.10.1/src/lib.rs:38:1

Maybe this code?
https://github.com/gakonst/foundry/blob/568534d2c44a1f93b59a1a3104fcd2c0bf5f23a9/forge/src/runner.rs#L685-L689

@alexeuler
Copy link
Contributor Author

We seem to have hit a Shrinking issue?



---- runner::tests::sputnik::test_fuzz_shrinking stdout ----
thread 'runner::tests::sputnik::test_fuzz_shrinking' panicked at 'Integer overflow when casting to u64', /home/runner/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/primitive-types-0.10.1/src/lib.rs:38:1

Maybe this code?

https://github.com/gakonst/foundry/blob/568534d2c44a1f93b59a1a3104fcd2c0bf5f23a9/forge/src/runner.rs#L685-L689

Yeah it's bacause I made dict and edge cases values not shrinkable. I reverted that for this time. Probably it makes sense to try shrink these values since iterations are run pretty fast anyway...

@gakonst gakonst merged commit 55b6b9a into foundry-rs:master Feb 15, 2022
@gakonst
Copy link
Member

gakonst commented Feb 15, 2022

@alexeuler this is working great inc ombination with the Assume PR!

gakonst added a commit to gakonst/solmate that referenced this pull request Feb 16, 2022
Otherwise they can be invalid curve points since Secp256k1's Curve Order is `0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141`.
See [k256](https://docs.rs/k256/latest/src/k256/lib.rs.html#54)'s implemenation.

This was detected in Forge after we made the fuzzer smarter
foundry-rs/foundry#735
@alexeuler alexeuler mentioned this pull request Jul 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-feature Type: feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants