C - read vs mmap vs fread

Read and mmap are both fundamentally important system calls, used to access bytes in files. Read uses the standard file descriptor to access files while mmap maps files to RAM. The purpose of this experiment was to compare performance differences with read when compared to mmap.

Approach

1. Randomly generate file content for different file sizes.
2. Calculate file histogram and measure elapsed time - 10 times
3. Write elapsed time, average time and standard deviation to a file.

These tests were conducted on a Ubuntu 20.04.6 LTS system utilizing an Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz × 6 with 9.4.0 gcc version.

Getting started

Clone the project or download it as a zip and then unzip it.

Conduct standard test

Run

bash script.sh -h

and then use the appropriate flag.

Conduct custom test

Run

make
./main -h

and then use the appropriate flag.

Note: you can firstly use bash script.sh -c to generate test files.

Results

Buffer sizes are in bytes, time is in milliseconds.

File size: 512B

512B	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	0.0039	0.0033	0.0034	0.0045	0.0061	0.0074	0.0429	0.045	0.0443	0.0473	0.0476
Standard deviation	0.0036	0.0038	0.0034	0.0037	0.0035	0.0045	0.0046	0.0051	0.0049	0.0061	0.0058

Better method: read(1024)

---

File size: 2kB

2kB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	0.0058	0.0053	0.0044	0.0055	0.0075	0.0088	0.0467	0.0434	0.0447	0.0495	0.0466
Standard deviation	0.004	0.0038	0.0037	0.0044	0.0039	0.0061	0.0044	0.0035	0.004	0.011	0.0041

Better method: read(4096)

---

File size: 8kB

8kB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	0.0144	0.0118	0.01	0.0107	0.0117	0.0124	0.0468	0.044	0.0444	0.0485	0.0515
Standard deviation	0.0059	0.0047	0.0046	0.0054	0.0047	0.0066	0.0082	0.0054	0.0048	0.0047	0.0046

Better method: read(4096)

---

File size: 32kB

32kB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	0.0466	0.0398	0.0297	0.0298	0.0302	0.0288	0.044	0.0439	0.0467	0.0499	0.0569
Standard deviation	0.0072	0.0117	0.0073	0.0082	0.0056	0.0071	0.0063	0.005	0.0069	0.0049	0.0063

Better method: mmap

---

File size: 128kB

128kB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	0.185	0.1407	0.1154	0.1056	0.1089	0.0927	0.0478	0.0441	0.0451	0.0505	0.0591
Standard deviation	0.0167	0.0134	0.0179	0.0145	0.0152	0.0145	0.0063	0.005	0.0049	0.0066	0.0088

Better method: fread(1024)

---

File size: 512kB

512kB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	0.7137	0.5605	0.4293	0.4132	0.4039	0.3593	0.0446	0.045	0.0457	0.0486	0.0637
Standard deviation	0.0387	0.0462	0.0352	0.0493	0.0324	0.0911	0.0053	0.0046	0.0049	0.0053	0.0088

Better method: fread(512)

---

File size: 1MB

1MB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	1.4308	1.1021	0.8901	0.8104	0.8133	0.6797	0.0464	0.0451	0.0462	0.0499	0.0642
Standard deviation	0.0771	0.0656	0.0668	0.077	0.0673	0.0797	0.0062	0.0102	0.0074	0.0059	0.0148

Better method: fread(1024)

---

File size: 8MB

8MB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	11.2172	8.8375	7.0907	6.6529	6.4918	5.1665	0.0454	0.0434	0.0439	0.0483	0.059
Standard deviation	0.6225	0.4995	0.4453	0.5361	0.5419	0.5467	0.0058	0.0049	0.0049	0.0059	0.0094

Better method: fread(1024)

---

File size: 32MB

32MB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	44.5635	35.1965	28.2577	26.4077	25.6555	20.9024	0.0442	0.069	0.0473	0.0521	0.0563
Standard deviation	2.3593	1.8763	1.9638	2.1978	1.8554	1.9359	0.0052	0.0261	0.0065	0.0115	0.0065

Better method: fread(512)

---

File size: 256MB

256MB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	350.7526	289.6301	259.0249	237.116	207.8174	170.2994	0.0461	0.0441	0.0495	0.0533	0.0568
Standard deviation	14.843	19.1331	30.4049	41.2148	20.9232	16.17	0.0141	0.0047	0.007	0.0072	0.0059

Better method: fread(1024)

---

File size: 1GB

1GB	read(512)	read(1024)	read(4096)	read(8192)	read(16384)	mmap	fread(512)	fread(1024)	fread(4096)	fread(8192)	fread(16384)
Average	1390.0621	1104.0805	981.7573	896.7686	876.6501	728.8994	0.0498	0.0439	0.0475	0.0496	0.0559
Standard deviation	92.2652	50.4362	90.6253	89.6159	59.6426	57.3032	0.0124	0.0057	0.0073	0.0059	0.0064

Better method: fread(1024)

Conclusion

For smaller files, the read method is faster when compared to the mmap and fread methods. That changes with large files - mmap is faster than read, and fread is faster than mmap.

For smaller files, the most appropriate buffer size would be 4096B or 8192B, which is the same as a page size on the computer used in the experiment. The page size can be found by running getconf PAGE_SIZE.

For larger files, the most suitable buffer size would be 16kB.