Attached patch implements an O_STREAMING file I/O flag which enables
manual drop-behind of pages.
If the file has O_STREAMING set then the user has explicitly said "this
is streaming data, I know I will not revisit this, do not cache
anything". So we drop pages from the pagecache before our current
index. We have to fiddle a bit to get writes working since we do
write-behind but the logic is there and it works.
Some numbers. A simple streaming read to verify the pagecache effects:
Streaming 1GB Read (avg of many runs, mem=2GB):
O_STREAMING Wall time Change in Page Cache
Yes 25.58s 0
No 25.55s +835MB
Another read with much more VM pressure:
Streaming 1GB Read (avg of many runs, mem=8M)
O_STREAMING Wall time Change in Page Cache
Yes 25.76s 0
No 29.01s +1MB
And now the kicker:
Kernel compile (make -j2) and concurrent streaming I/O
(avg of two runs, mem=128M):
O_STREAMING Time to complete Kernel Compile
Yes 3m27.863s
No 4m15.818s
This is c/o Andrew Morton.