android/android_kernel_samsung_sm8650
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge 8834147f95 ("Merge tag 'fscache-rewrite-20220111' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs") into android-mainline

Browse Source 
Steps on the way to 5.17-rc1

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I6904ae6aad6acb8e56092d1e941cd73039ce0a63
This commit is contained in:
Greg Kroah-Hartman 2022-01-31 20:13:39 +01:00
commit 0484cf9136
93 changed files with 7610 additions and 13406 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

954
Documentation/filesystems/caching/backend-api.rst
View File

File diff suppressed because it is too large Load Diff

6
Documentation/filesystems/caching/cachefiles.rst
View File

@ -1,8 +1,8 @@
.. SPDX-License-Identifier: GPL-2.0
===============================================
CacheFiles: CACHE ON ALREADY MOUNTED FILESYSTEM
===============================================
===================================
Cache on Already Mounted Filesystem
===================================
.. Contents:

519
Documentation/filesystems/caching/fscache.rst
View File

@ -10,25 +10,25 @@ Overview
This facility is a general purpose cache for network filesystems, though it
could be used for caching other things such as ISO9660 filesystems too.
FS-Cache mediates between cache backends (such as CacheFS) and network
FS-Cache mediates between cache backends (such as CacheFiles) and network
filesystems::
+---------+
| | +--------------+
| NFS |--+ | |
| | | +-->| CacheFS |
+---------+ | +----------+ | | /dev/hda5 |
| | | | +--------------+
+---------+ +-->| | |
| | | |--+
| AFS |----->| FS-Cache |
| | | |--+
+---------+ +-->| | |
| | | | +--------------+
+---------+ | +----------+ | | |
| | | +-->| CacheFiles |
| ISOFS |--+ | /var/cache |
| | +--------------+
| | +--------------+
| NFS |--+ | |
| | | +-->| CacheFS |
+---------+ | +----------+ | | /dev/hda5 |
| | | | +--------------+
+---------+ +-------------->| | |
| | +-------+ | |--+
| AFS |----->| | | FS-Cache |
| | | netfs |-->| |--+
+---------+ +-->| lib | | | |
| | | | | | +--------------+
+---------+ | +-------+ +----------+ | | |
| | | +-->| CacheFiles |
| 9P |--+ | /var/cache |
| | +--------------+
+---------+
Or to look at it another way, FS-Cache is a module that provides a caching
@ -84,101 +84,62 @@ then serving the pages out of that cache rather than the netfs inode because:
one-off access of a small portion of it (such as might be done with the
"file" program).
It instead serves the cache out in PAGE_SIZE chunks as and when requested by
the netfs('s) using it.
It instead serves the cache out in chunks as and when requested by the netfs
using it.
FS-Cache provides the following facilities:
(1) More than one cache can be used at once. Caches can be selected
* More than one cache can be used at once. Caches can be selected
explicitly by use of tags.
(2) Caches can be added / removed at any time.
* Caches can be added / removed at any time, even whilst being accessed.
(3) The netfs is provided with an interface that allows either party to
* The netfs is provided with an interface that allows either party to
withdraw caching facilities from a file (required for (2)).
(4) The interface to the netfs returns as few errors as possible, preferring
* The interface to the netfs returns as few errors as possible, preferring
rather to let the netfs remain oblivious.
(5) Cookies are used to represent indices, files and other objects to the
netfs. The simplest cookie is just a NULL pointer - indicating nothing
cached there.
* There are three types of cookie: cache, volume and data file cookies.
Cache cookies represent the cache as a whole and are not normally visible
to the netfs; the netfs gets a volume cookie to represent a collection of
files (typically something that a netfs would get for a superblock); and
data file cookies are used to cache data (something that would be got for
an inode).
(6) The netfs is allowed to propose - dynamically - any index hierarchy it
desires, though it must be aware that the index search function is
recursive, stack space is limited, and indices can only be children of
indices.
* Volumes are matched using a key. This is a printable string that is used
to encode all the information that might be needed to distinguish one
superblock, say, from another. This would be a compound of things like
cell name or server address, volume name or share path. It must be a
valid pathname.
(7) Data I/O is done direct to and from the netfs's pages. The netfs
indicates that page A is at index B of the data-file represented by cookie
C, and that it should be read or written. The cache backend may or may
not start I/O on that page, but if it does, a netfs callback will be
invoked to indicate completion. The I/O may be either synchronous or
asynchronous.
* Cookies are matched using a key. This is a binary blob and is used to
represent the object within a volume (so the volume key need not form
part of the blob). This might include things like an inode number and
uniquifier or a file handle.
(8) Cookies can be "retired" upon release. At this point FS-Cache will mark
them as obsolete and the index hierarchy rooted at that point will get
recycled.
* Cookie resources are set up and pinned by marking the cookie in-use.
This prevents the backing resources from being culled. Timed garbage
collection is employed to eliminate cookies that haven't been used for a
short while, thereby reducing resource overload. This is intended to be
used when a file is opened or closed.
(9) The netfs provides a "match" function for index searches. In addition to
saying whether a match was made or not, this can also specify that an
entry should be updated or deleted.
A cookie can be marked in-use multiple times simultaneously; each mark
must be unused.
(10) As much as possible is done asynchronously.
* Begin/end access functions are provided to delay cache withdrawal for the
duration of an operation and prevent structs from being freed whilst
we're looking at them.
* Data I/O is done by asynchronous DIO to/from a buffer described by the
netfs using an iov_iter.
FS-Cache maintains a virtual indexing tree in which all indices, files, objects
and pages are kept. Bits of this tree may actually reside in one or more
caches::
* An invalidation facility is available to discard data from the cache and
to deal with I/O that's in progress that is accessing old data.
FSDEF
|
+------------------------------------+
| |
NFS AFS
| |
+--------------------------+ +-----------+
| | | |
homedir mirror afs.org redhat.com
| | |
+------------+ +---------------+ +----------+
| | | | | |
00001 00002 00007 00125 vol00001 vol00002
| | | | |
+---+---+ +-----+ +---+ +------+------+ +-----+----+
| | | | | | | | | | | | |
PG0 PG1 PG2 PG0 XATTR PG0 PG1 DIRENT DIRENT DIRENT R/W R/O Bak
| |
PG0 +-------+
| |
00001 00003
|
+---+---+
| | |
PG0 PG1 PG2
In the example above, you can see two netfs's being backed: NFS and AFS. These
have different index hierarchies:
* The NFS primary index contains per-server indices. Each server index is
indexed by NFS file handles to get data file objects. Each data file
objects can have an array of pages, but may also have further child
objects, such as extended attributes and directory entries. Extended
attribute objects themselves have page-array contents.
* The AFS primary index contains per-cell indices. Each cell index contains
per-logical-volume indices. Each of volume index contains up to three
indices for the read-write, read-only and backup mirrors of those volumes.
Each of these contains vnode data file objects, each of which contains an
array of pages.
The very top index is the FS-Cache master index in which individual netfs's
have entries.
Any index object may reside in more than one cache, provided it only has index
children. Any index with non-index object children will be assumed to only
reside in one cache.
* Cookies can be "retired" upon release, thereby causing the object to be
removed from the cache.
The netfs API to FS-Cache can be found in:
@ -189,11 +150,6 @@ The cache backend API to FS-Cache can be found in:
Documentation/filesystems/caching/backend-api.rst
A description of the internal representations and object state machine can be
found in:
Documentation/filesystems/caching/object.rst
Statistical Information
=======================
@ -201,333 +157,162 @@ Statistical Information
If FS-Cache is compiled with the following options enabled::
CONFIG_FSCACHE_STATS=y
CONFIG_FSCACHE_HISTOGRAM=y
then it will gather certain statistics and display them through a number of
proc files.
then it will gather certain statistics and display them through:
/proc/fs/fscache/stats
----------------------
/proc/fs/fscache/stats
This shows counts of a number of events that can happen in FS-Cache:
This shows counts of a number of events that can happen in FS-Cache:
+--------------+-------+-------------------------------------------------------+
|CLASS |EVENT |MEANING |
+==============+=======+=======================================================+
|Cookies |idx=N |Number of index cookies allocated |
|Cookies |n=N |Number of data storage cookies allocated |
+ +-------+-------------------------------------------------------+
| |dat=N |Number of data storage cookies allocated |
| |v=N |Number of volume index cookies allocated |
+ +-------+-------------------------------------------------------+
| |spc=N |Number of special cookies allocated |
+--------------+-------+-------------------------------------------------------+
|Objects |alc=N |Number of objects allocated |
| |vcol=N |Number of volume index key collisions |
+ +-------+-------------------------------------------------------+
| |nal=N |Number of object allocation failures |
+ +-------+-------------------------------------------------------+
| |avl=N |Number of objects that reached the available state |
+ +-------+-------------------------------------------------------+
| |ded=N |Number of objects that reached the dead state |
+--------------+-------+-------------------------------------------------------+
|ChkAux |non=N |Number of objects that didn't have a coherency check |
+ +-------+-------------------------------------------------------+
| |ok=N |Number of objects that passed a coherency check |
+ +-------+-------------------------------------------------------+
| |upd=N |Number of objects that needed a coherency data update |
+ +-------+-------------------------------------------------------+
| |obs=N |Number of objects that were declared obsolete |
+--------------+-------+-------------------------------------------------------+
|Pages |mrk=N |Number of pages marked as being cached |
| |unc=N |Number of uncache page requests seen |
| |voom=N |Number of OOM events when allocating volume cookies |
+--------------+-------+-------------------------------------------------------+
|Acquire |n=N |Number of acquire cookie requests seen |
+ +-------+-------------------------------------------------------+
| |nul=N |Number of acq reqs given a NULL parent |
+ +-------+-------------------------------------------------------+
| |noc=N |Number of acq reqs rejected due to no cache available |
+ +-------+-------------------------------------------------------+
| |ok=N |Number of acq reqs succeeded |
+ +-------+-------------------------------------------------------+
| |nbf=N |Number of acq reqs rejected due to error |
+ +-------+-------------------------------------------------------+
| |oom=N |Number of acq reqs failed on ENOMEM |
+--------------+-------+-------------------------------------------------------+
|Lookups |n=N |Number of lookup calls made on cache backends |
|LRU |n=N |Number of cookies currently on the LRU |
+ +-------+-------------------------------------------------------+
| |neg=N |Number of negative lookups made |
| |exp=N |Number of cookies expired off of the LRU |
+ +-------+-------------------------------------------------------+
| |pos=N |Number of positive lookups made |
| |rmv=N |Number of cookies removed from the LRU |
+ +-------+-------------------------------------------------------+
| |crt=N |Number of objects created by lookup |
| |drp=N |Number of LRU'd cookies relinquished/withdrawn |
+ +-------+-------------------------------------------------------+
| |tmo=N |Number of lookups timed out and requeued |
| |at=N |Time till next LRU cull (jiffies) |
+--------------+-------+-------------------------------------------------------+
|Invals |n=N |Number of invalidations |
+--------------+-------+-------------------------------------------------------+
|Updates |n=N |Number of update cookie requests seen |
+ +-------+-------------------------------------------------------+
| |nul=N |Number of upd reqs given a NULL parent |
| |rsz=N |Number of resize requests |
+ +-------+-------------------------------------------------------+
| |run=N |Number of upd reqs granted CPU time |
| |rsn=N |Number of skipped resize requests |
+--------------+-------+-------------------------------------------------------+
|Relinqs |n=N |Number of relinquish cookie requests seen |
+ +-------+-------------------------------------------------------+
| |nul=N |Number of rlq reqs given a NULL parent |
| |rtr=N |Number of rlq reqs with retire=true |
+ +-------+-------------------------------------------------------+
| |wcr=N |Number of rlq reqs waited on completion of creation |
| |drop=N |Number of cookies no longer blocking re-acquisition |
+--------------+-------+-------------------------------------------------------+
|AttrChg |n=N |Number of attribute changed requests seen |
|NoSpace |nwr=N |Number of write requests refused due to lack of space |
+ +-------+-------------------------------------------------------+
| |ok=N |Number of attr changed requests queued |
| |ncr=N |Number of create requests refused due to lack of space |
+ +-------+-------------------------------------------------------+
| |nbf=N |Number of attr changed rejected -ENOBUFS |
+ +-------+-------------------------------------------------------+
| |oom=N |Number of attr changed failed -ENOMEM |
+ +-------+-------------------------------------------------------+
| |run=N |Number of attr changed ops given CPU time |
| |cull=N |Number of objects culled to make space |
+--------------+-------+-------------------------------------------------------+
|Allocs |n=N |Number of allocation requests seen |
|IO |rd=N |Number of read operations in the cache |
+ +-------+-------------------------------------------------------+
| |ok=N |Number of successful alloc reqs |
+ +-------+-------------------------------------------------------+
| |wt=N |Number of alloc reqs that waited on lookup completion |
+ +-------+-------------------------------------------------------+
| |nbf=N |Number of alloc reqs rejected -ENOBUFS |
+ +-------+-------------------------------------------------------+
| |int=N |Number of alloc reqs aborted -ERESTARTSYS |
+ +-------+-------------------------------------------------------+
| |ops=N |Number of alloc reqs submitted |
+ +-------+-------------------------------------------------------+
| |owt=N |Number of alloc reqs waited for CPU time |
+ +-------+-------------------------------------------------------+
| |abt=N |Number of alloc reqs aborted due to object death |
+--------------+-------+-------------------------------------------------------+
|Retrvls |n=N |Number of retrieval (read) requests seen |
+ +-------+-------------------------------------------------------+
| |ok=N |Number of successful retr reqs |
+ +-------+-------------------------------------------------------+
| |wt=N |Number of retr reqs that waited on lookup completion |
+ +-------+-------------------------------------------------------+
| |nod=N |Number of retr reqs returned -ENODATA |
+ +-------+-------------------------------------------------------+
| |nbf=N |Number of retr reqs rejected -ENOBUFS |
+ +-------+-------------------------------------------------------+
| |int=N |Number of retr reqs aborted -ERESTARTSYS |
+ +-------+-------------------------------------------------------+
| |oom=N |Number of retr reqs failed -ENOMEM |
+ +-------+-------------------------------------------------------+
| |ops=N |Number of retr reqs submitted |
+ +-------+-------------------------------------------------------+
| |owt=N |Number of retr reqs waited for CPU time |
+ +-------+-------------------------------------------------------+
| |abt=N |Number of retr reqs aborted due to object death |
+--------------+-------+-------------------------------------------------------+
|Stores |n=N |Number of storage (write) requests seen |
+ +-------+-------------------------------------------------------+
| |ok=N |Number of successful store reqs |
+ +-------+-------------------------------------------------------+
| |agn=N |Number of store reqs on a page already pending storage |
+ +-------+-------------------------------------------------------+
| |nbf=N |Number of store reqs rejected -ENOBUFS |
+ +-------+-------------------------------------------------------+
| |oom=N |Number of store reqs failed -ENOMEM |
+ +-------+-------------------------------------------------------+
| |ops=N |Number of store reqs submitted |
+ +-------+-------------------------------------------------------+
| |run=N |Number of store reqs granted CPU time |
+ +-------+-------------------------------------------------------+
| |pgs=N |Number of pages given store req processing time |
+ +-------+-------------------------------------------------------+
| |rxd=N |Number of store reqs deleted from tracking tree |
+ +-------+-------------------------------------------------------+
| |olm=N |Number of store reqs over store limit |
+--------------+-------+-------------------------------------------------------+
|VmScan |nos=N |Number of release reqs against pages with no |
| | |pending store |
+ +-------+-------------------------------------------------------+
| |gon=N |Number of release reqs against pages stored by |
| | |time lock granted |
+ +-------+-------------------------------------------------------+
| |bsy=N |Number of release reqs ignored due to in-progress store|
+ +-------+-------------------------------------------------------+
| |can=N |Number of page stores cancelled due to release req |
+--------------+-------+-------------------------------------------------------+
|Ops |pend=N |Number of times async ops added to pending queues |
+ +-------+-------------------------------------------------------+
| |run=N |Number of times async ops given CPU time |
+ +-------+-------------------------------------------------------+
| |enq=N |Number of times async ops queued for processing |
+ +-------+-------------------------------------------------------+
| |can=N |Number of async ops cancelled |
+ +-------+-------------------------------------------------------+
| |rej=N |Number of async ops rejected due to object |
| | |lookup/create failure |
+ +-------+-------------------------------------------------------+
| |ini=N |Number of async ops initialised |
+ +-------+-------------------------------------------------------+
| |dfr=N |Number of async ops queued for deferred release |
+ +-------+-------------------------------------------------------+
| |rel=N |Number of async ops released |
| | |(should equal ini=N when idle) |
+ +-------+-------------------------------------------------------+
| |gc=N |Number of deferred-release async ops garbage collected |
+--------------+-------+-------------------------------------------------------+
|CacheOp |alo=N |Number of in-progress alloc_object() cache ops |
+ +-------+-------------------------------------------------------+
| |luo=N |Number of in-progress lookup_object() cache ops |
+ +-------+-------------------------------------------------------+
| |luc=N |Number of in-progress lookup_complete() cache ops |
+ +-------+-------------------------------------------------------+
| |gro=N |Number of in-progress grab_object() cache ops |
+ +-------+-------------------------------------------------------+
| |upo=N |Number of in-progress update_object() cache ops |
+ +-------+-------------------------------------------------------+
| |dro=N |Number of in-progress drop_object() cache ops |
+ +-------+-------------------------------------------------------+
| |pto=N |Number of in-progress put_object() cache ops |
+ +-------+-------------------------------------------------------+
| |syn=N |Number of in-progress sync_cache() cache ops |
+ +-------+-------------------------------------------------------+
| |atc=N |Number of in-progress attr_changed() cache ops |
+ +-------+-------------------------------------------------------+
| |rap=N |Number of in-progress read_or_alloc_page() cache ops |
+ +-------+-------------------------------------------------------+
| |ras=N |Number of in-progress read_or_alloc_pages() cache ops |
+ +-------+-------------------------------------------------------+
| |alp=N |Number of in-progress allocate_page() cache ops |
+ +-------+-------------------------------------------------------+
| |als=N |Number of in-progress allocate_pages() cache ops |
+ +-------+-------------------------------------------------------+
| |wrp=N |Number of in-progress write_page() cache ops |
+ +-------+-------------------------------------------------------+
| |ucp=N |Number of in-progress uncache_page() cache ops |
+ +-------+-------------------------------------------------------+
| |dsp=N |Number of in-progress dissociate_pages() cache ops |
+--------------+-------+-------------------------------------------------------+
|CacheEv |nsp=N |Number of object lookups/creations rejected due to |
| | |lack of space |
+ +-------+-------------------------------------------------------+
| |stl=N |Number of stale objects deleted |
+ +-------+-------------------------------------------------------+
| |rtr=N |Number of objects retired when relinquished |
+ +-------+-------------------------------------------------------+
| |cul=N |Number of objects culled |
| |wr=N |Number of write operations in the cache |
+--------------+-------+-------------------------------------------------------+
Netfslib will also add some stats counters of its own.
/proc/fs/fscache/histogram
--------------------------
Cache List
==========
::
FS-Cache provides a list of cache cookies:
cat /proc/fs/fscache/histogram
JIFS SECS OBJ INST OP RUNS OBJ RUNS RETRV DLY RETRIEVLS
===== ===== ========= ========= ========= ========= =========
This shows the breakdown of the number of times each amount of time
between 0 jiffies and HZ-1 jiffies a variety of tasks took to run. The
columns are as follows:
========= =======================================================
COLUMN TIME MEASUREMENT
========= =======================================================
OBJ INST Length of time to instantiate an object
OP RUNS Length of time a call to process an operation took
OBJ RUNS Length of time a call to process an object event took
RETRV DLY Time between an requesting a read and lookup completing
RETRIEVLS Time between beginning and end of a retrieval
========= =======================================================
Each row shows the number of events that took a particular range of times.
Each step is 1 jiffy in size. The JIFS column indicates the particular
jiffy range covered, and the SECS field the equivalent number of seconds.
Object List
===========
If CONFIG_FSCACHE_OBJECT_LIST is enabled, the FS-Cache facility will maintain a
list of all the objects currently allocated and allow them to be viewed
through::
/proc/fs/fscache/objects
/proc/fs/fscache/cookies
This will look something like::
[root@andromeda ~]# head /proc/fs/fscache/objects
OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS EM EV F S | NETFS_COOKIE_DEF TY FL NETFS_DATA OBJECT_KEY, AUX_DATA
======== ======== ==== ===== === === === == ===== == == = = | ================ == == ================ ================
17e4b 2 ACTV 0 0 0 0 0 0 7b 4 0 0 | NFS.fh DT 0 ffff88001dd82820 010006017edcf8bbc93b43298fdfbe71e50b57b13a172c0117f38472, e567634700000000000000000000000063f2404a000000000000000000000000c9030000000000000000000063f2404a
1693a 2 ACTV 0 0 0 0 0 0 7b 4 0 0 | NFS.fh DT 0 ffff88002db23380 010006017edcf8bbc93b43298fdfbe71e50b57b1e0162c01a2df0ea6, 420ebc4a000000000000000000000000420ebc4a0000000000000000000000000e1801000000000000000000420ebc4a
# cat /proc/fs/fscache/caches
CACHE REF VOLS OBJS ACCES S NAME
======== ===== ===== ===== ===== = ===============
00000001 2 1 2123 1 A default
where the first set of columns before the '|' describe the object:
where the columns are:
======= ===============================================================
COLUMN DESCRIPTION
======= ===============================================================
OBJECT Object debugging ID (appears as OBJ%x in some debug messages)
PARENT Debugging ID of parent object
STAT Object state
CHLDN Number of child objects of this object
OPS Number of outstanding operations on this object
OOP Number of outstanding child object management operations
IPR
EX Number of outstanding exclusive operations
READS Number of outstanding read operations
EM Object's event mask
EV Events raised on this object
F Object flags
S Object work item busy state mask (1:pending 2:running)
CACHE Cache cookie debug ID (also appears in traces)
REF Number of references on the cache cookie
VOLS Number of volumes cookies in this cache
OBJS Number of cache objects in use
ACCES Number of accesses pinning the cache
S State
NAME Name of the cache.
======= ===============================================================
and the second set of columns describe the object's cookie, if present:
The state can be (-) Inactive, (P)reparing, (A)ctive, (E)rror or (W)ithdrawing.
================ ======================================================
COLUMN DESCRIPTION
================ ======================================================
NETFS_COOKIE_DEF Name of netfs cookie definition
TY Cookie type (IX - index, DT - data, hex - special)
FL Cookie flags
NETFS_DATA Netfs private data stored in the cookie
OBJECT_KEY Object key } 1 column, with separating comma
AUX_DATA Object aux data } presence may be configured
================ ======================================================
The data shown may be filtered by attaching the a key to an appropriate keyring
before viewing the file. Something like::
Volume List
===========
keyctl add user fscache:objlist <restrictions> @s
FS-Cache provides a list of volume cookies:
where <restrictions> are a selection of the following letters:
/proc/fs/fscache/volumes
== =========================================================
K Show hexdump of object key (don't show if not given)
A Show hexdump of object aux data (don't show if not given)
== =========================================================
This will look something like::
and the following paired letters:
VOLUME REF nCOOK ACC FL CACHE KEY
======== ===== ===== === == =============== ================
00000001 55 54 1 00 default afs,example.com,100058
== =========================================================
C Show objects that have a cookie
c Show objects that don't have a cookie
B Show objects that are busy
b Show objects that aren't busy
W Show objects that have pending writes
w Show objects that don't have pending writes
R Show objects that have outstanding reads
r Show objects that don't have outstanding reads
S Show objects that have work queued
s Show objects that don't have work queued
== =========================================================
where the columns are:
If neither side of a letter pair is given, then both are implied. For example:
======= ===============================================================
COLUMN DESCRIPTION
======= ===============================================================
VOLUME The volume cookie debug ID (also appears in traces)
REF Number of references on the volume cookie
nCOOK Number of cookies in the volume
ACC Number of accesses pinning the cache
FL Flags on the volume cookie
CACHE Name of the cache or "-"
KEY The indexing key for the volume
======= ===============================================================
keyctl add user fscache:objlist KB @s
shows objects that are busy, and lists their object keys, but does not dump
their auxiliary data. It also implies "CcWwRrSs", but as 'B' is given, 'b' is
not implied.
Cookie List
===========
By default all objects and all fields will be shown.
FS-Cache provides a list of cookies:
/proc/fs/fscache/cookies
This will look something like::
# head /proc/fs/fscache/cookies
COOKIE VOLUME REF ACT ACC S FL DEF
======== ======== === === === = == ================
00000435 00000001 1 0 -1 - 08 0000000201d080070000000000000000, 0000000000000000
00000436 00000001 1 0 -1 - 00 0000005601d080080000000000000000, 0000000000000051
00000437 00000001 1 0 -1 - 08 00023b3001d0823f0000000000000000, 0000000000000000
00000438 00000001 1 0 -1 - 08 0000005801d0807b0000000000000000, 0000000000000000
00000439 00000001 1 0 -1 - 08 00023b3201d080a10000000000000000, 0000000000000000
0000043a 00000001 1 0 -1 - 08 00023b3401d080a30000000000000000, 0000000000000000
0000043b 00000001 1 0 -1 - 08 00023b3601d080b30000000000000000, 0000000000000000
0000043c 00000001 1 0 -1 - 08 00023b3801d080b40000000000000000, 0000000000000000
where the columns are:
======= ===============================================================
COLUMN DESCRIPTION
======= ===============================================================
COOKIE The cookie debug ID (also appears in traces)
VOLUME The parent volume cookie debug ID
REF Number of references on the volume cookie
ACT Number of times the cookie is marked for in use
ACC Number of access pins in the cookie
S State of the cookie
FL Flags on the cookie
DEF Key, auxiliary data
======= ===============================================================
Debugging
@ -549,10 +334,8 @@ This is a bitmask of debugging streams to enable:
3 8 Cookie management Function entry trace
4 16 Function exit trace
5 32 General
6 64 Page handling Function entry trace
7 128 Function exit trace
8 256 General
9 512 Operation management Function entry trace
6-8 (Not used)
9 512 I/O operation management Function entry trace
10 1024 Function exit trace
11 2048 General
======= ======= =============================== =======================
@ -560,6 +343,6 @@ This is a bitmask of debugging streams to enable:
The appropriate set of values should be OR'd together and the result written to
the control file. For example::
echo $((1|8|64)) >/sys/module/fscache/parameters/debug
echo $((1|8|512)) >/sys/module/fscache/parameters/debug
will turn on all function entry debugging.

4
Documentation/filesystems/caching/index.rst
View File

@ -7,8 +7,6 @@ Filesystem Caching
:maxdepth: 2
fscache
object
netfs-api
backend-api
cachefiles
netfs-api
operations

1264
Documentation/filesystems/caching/netfs-api.rst
View File

File diff suppressed because it is too large Load Diff

313
Documentation/filesystems/caching/object.rst
View File

@ -1,313 +0,0 @@
.. SPDX-License-Identifier: GPL-2.0
====================================================
In-Kernel Cache Object Representation and Management
====================================================
By: David Howells <dhowells@redhat.com>
.. Contents:
(*) Representation
(*) Object management state machine.
- Provision of cpu time.
- Locking simplification.
(*) The set of states.
(*) The set of events.
Representation
==============
FS-Cache maintains an in-kernel representation of each object that a netfs is
currently interested in. Such objects are represented by the fscache_cookie
struct and are referred to as cookies.
FS-Cache also maintains a separate in-kernel representation of the objects that
a cache backend is currently actively caching. Such objects are represented by
the fscache_object struct. The cache backends allocate these upon request, and
are expected to embed them in their own representations. These are referred to
as objects.
There is a 1:N relationship between cookies and objects. A cookie may be
represented by multiple objects - an index may exist in more than one cache -
or even by no objects (it may not be cached).
Furthermore, both cookies and objects are hierarchical. The two hierarchies
correspond, but the cookies tree is a superset of the union of the object trees
of multiple caches::
NETFS INDEX TREE : CACHE 1 : CACHE 2
: :
: +-----------+ :
+----------->| IObject | :
+-----------+ | : +-----------+ :
| ICookie |-------+ : | :
+-----------+ | : | : +-----------+
| +------------------------------>| IObject |
| : | : +-----------+
| : V : |
| : +-----------+ : |
V +----------->| IObject | : |
+-----------+ | : +-----------+ : |
| ICookie |-------+ : | : V
+-----------+ | : | : +-----------+
| +------------------------------>| IObject |
+-----+-----+ : | : +-----------+
| | : | : |
V | : V : |
+-----------+ | : +-----------+ : |
| ICookie |------------------------->| IObject | : |
+-----------+ | : +-----------+ : |
| V : | : V
| +-----------+ : | : +-----------+
| | ICookie |-------------------------------->| IObject |
| +-----------+ : | : +-----------+
V | : V : |
+-----------+ | : +-----------+ : |
| DCookie |------------------------->| DObject | : |
+-----------+ | : +-----------+ : |
| : : |
+-------+-------+ : : |
| | : : |
V V : : V
+-----------+ +-----------+ : : +-----------+
| DCookie | | DCookie |------------------------>| DObject |
+-----------+ +-----------+ : : +-----------+
: :
In the above illustration, ICookie and IObject represent indices and DCookie
and DObject represent data storage objects. Indices may have representation in
multiple caches, but currently, non-index objects may not. Objects of any type
may also be entirely unrepresented.
As far as the netfs API goes, the netfs is only actually permitted to see
pointers to the cookies. The cookies themselves and any objects attached to
those cookies are hidden from it.
Object Management State Machine
===============================
Within FS-Cache, each active object is managed by its own individual state
machine. The state for an object is kept in the fscache_object struct, in
object->state. A cookie may point to a set of objects that are in different
states.
Each state has an action associated with it that is invoked when the machine
wakes up in that state. There are four logical sets of states:
(1) Preparation: states that wait for the parent objects to become ready. The
representations are hierarchical, and it is expected that an object must
be created or accessed with respect to its parent object.
(2) Initialisation: states that perform lookups in the cache and validate
what's found and that create on disk any missing metadata.
(3) Normal running: states that allow netfs operations on objects to proceed
and that update the state of objects.
(4) Termination: states that detach objects from their netfs cookies, that
delete objects from disk, that handle disk and system errors and that free
up in-memory resources.
In most cases, transitioning between states is in response to signalled events.
When a state has finished processing, it will usually set the mask of events in
which it is interested (object->event_mask) and relinquish the worker thread.
Then when an event is raised (by calling fscache_raise_event()), if the event
is not masked, the object will be queued for processing (by calling
fscache_enqueue_object()).
Provision of CPU Time
---------------------
The work to be done by the various states was given CPU time by the threads of
the slow work facility. This was used in preference to the workqueue facility
because:
(1) Threads may be completely occupied for very long periods of time by a
particular work item. These state actions may be doing sequences of
synchronous, journalled disk accesses (lookup, mkdir, create, setxattr,
getxattr, truncate, unlink, rmdir, rename).
(2) Threads may do little actual work, but may rather spend a lot of time
sleeping on I/O. This means that single-threaded and 1-per-CPU-threaded
workqueues don't necessarily have the right numbers of threads.
Locking Simplification
----------------------
Because only one worker thread may be operating on any particular object's
state machine at once, this simplifies the locking, particularly with respect
to disconnecting the netfs's representation of a cache object (fscache_cookie)
from the cache backend's representation (fscache_object) - which may be
requested from either end.
The Set of States
=================
The object state machine has a set of states that it can be in. There are
preparation states in which the object sets itself up and waits for its parent
object to transit to a state that allows access to its children:
(1) State FSCACHE_OBJECT_INIT.
Initialise the object and wait for the parent object to become active. In
the cache, it is expected that it will not be possible to look an object
up from the parent object, until that parent object itself has been looked
up.
There are initialisation states in which the object sets itself up and accesses
disk for the object metadata:
(2) State FSCACHE_OBJECT_LOOKING_UP.
Look up the object on disk, using the parent as a starting point.
FS-Cache expects the cache backend to probe the cache to see whether this
object is represented there, and if it is, to see if it's valid (coherency
management).
The cache should call fscache_object_lookup_negative() to indicate lookup
failure for whatever reason, and should call fscache_obtained_object() to
indicate success.
At the completion of lookup, FS-Cache will let the netfs go ahead with
read operations, no matter whether the file is yet cached. If not yet
cached, read operations will be immediately rejected with ENODATA until
the first known page is uncached - as to that point there can be no data
to be read out of the cache for that file that isn't currently also held
in the pagecache.
(3) State FSCACHE_OBJECT_CREATING.
Create an object on disk, using the parent as a starting point. This
happens if the lookup failed to find the object, or if the object's
coherency data indicated what's on disk is out of date. In this state,
FS-Cache expects the cache to create
The cache should call fscache_obtained_object() if creation completes
successfully, fscache_object_lookup_negative() otherwise.
At the completion of creation, FS-Cache will start processing write
operations the netfs has queued for an object. If creation failed, the
write ops will be transparently discarded, and nothing recorded in the
cache.
There are some normal running states in which the object spends its time
servicing netfs requests:
(4) State FSCACHE_OBJECT_AVAILABLE.
A transient state in which pending operations are started, child objects
are permitted to advance from FSCACHE_OBJECT_INIT state, and temporary
lookup data is freed.
(5) State FSCACHE_OBJECT_ACTIVE.
The normal running state. In this state, requests the netfs makes will be
passed on to the cache.
(6) State FSCACHE_OBJECT_INVALIDATING.
The object is undergoing invalidation. When the state comes here, it
discards all pending read, write and attribute change operations as it is
going to clear out the cache entirely and reinitialise it. It will then
continue to the FSCACHE_OBJECT_UPDATING state.
(7) State FSCACHE_OBJECT_UPDATING.
The state machine comes here to update the object in the cache from the
netfs's records. This involves updating the auxiliary data that is used
to maintain coherency.
And there are terminal states in which an object cleans itself up, deallocates
memory and potentially deletes stuff from disk:
(8) State FSCACHE_OBJECT_LC_DYING.
The object comes here if it is dying because of a lookup or creation
error. This would be due to a disk error or system error of some sort.
Temporary data is cleaned up, and the parent is released.
(9) State FSCACHE_OBJECT_DYING.
The object comes here if it is dying due to an error, because its parent
cookie has been relinquished by the netfs or because the cache is being
withdrawn.
Any child objects waiting on this one are given CPU time so that they too
can destroy themselves. This object waits for all its children to go away
before advancing to the next state.
(10) State FSCACHE_OBJECT_ABORT_INIT.
The object comes to this state if it was waiting on its parent in
FSCACHE_OBJECT_INIT, but its parent died. The object will destroy itself
so that the parent may proceed from the FSCACHE_OBJECT_DYING state.
(11) State FSCACHE_OBJECT_RELEASING.
(12) State FSCACHE_OBJECT_RECYCLING.
The object comes to one of these two states when dying once it is rid of
all its children, if it is dying because the netfs relinquished its
cookie. In the first state, the cached data is expected to persist, and
in the second it will be deleted.
(13) State FSCACHE_OBJECT_WITHDRAWING.
The object transits to this state if the cache decides it wants to
withdraw the object from service, perhaps to make space, but also due to
error or just because the whole cache is being withdrawn.
(14) State FSCACHE_OBJECT_DEAD.
The object transits to this state when the in-memory object record is
ready to be deleted. The object processor shouldn't ever see an object in
this state.
The Set of Events
-----------------
There are a number of events that can be raised to an object state machine:
FSCACHE_OBJECT_EV_UPDATE
The netfs requested that an object be updated. The state machine will ask
the cache backend to update the object, and the cache backend will ask the
netfs for details of the change through its cookie definition ops.
FSCACHE_OBJECT_EV_CLEARED
This is signalled in two circumstances:
(a) when an object's last child object is dropped and
(b) when the last operation outstanding on an object is completed.
This is used to proceed from the dying state.
FSCACHE_OBJECT_EV_ERROR
This is signalled when an I/O error occurs during the processing of some
object.
FSCACHE_OBJECT_EV_RELEASE, FSCACHE_OBJECT_EV_RETIRE
These are signalled when the netfs relinquishes a cookie it was using.
The event selected depends on whether the netfs asks for the backing
object to be retired (deleted) or retained.
FSCACHE_OBJECT_EV_WITHDRAW
This is signalled when the cache backend wants to withdraw an object.
This means that the object will have to be detached from the netfs's
cookie.
Because the withdrawing releasing/retiring events are all handled by the object
state machine, it doesn't matter if there's a collision with both ends trying
to sever the connection at the same time. The state machine can just pick
which one it wants to honour, and that effects the other.

210
Documentation/filesystems/caching/operations.rst
View File

@ -1,210 +0,0 @@
.. SPDX-License-Identifier: GPL-2.0
================================
Asynchronous Operations Handling
================================
By: David Howells <dhowells@redhat.com>
.. Contents:
(*) Overview.
(*) Operation record initialisation.
(*) Parameters.
(*) Procedure.
(*) Asynchronous callback.
Overview
========
FS-Cache has an asynchronous operations handling facility that it uses for its
data storage and retrieval routines. Its operations are represented by
fscache_operation structs, though these are usually embedded into some other
structure.
This facility is available to and expected to be used by the cache backends,
and FS-Cache will create operations and pass them off to the appropriate cache
backend for completion.
To make use of this facility, <linux/fscache-cache.h> should be #included.
Operation Record Initialisation
===============================
An operation is recorded in an fscache_operation struct::
struct fscache_operation {
union {
struct work_struct fast_work;
struct slow_work slow_work;
};
unsigned long flags;
fscache_operation_processor_t processor;
...
};
Someone wanting to issue an operation should allocate something with this
struct embedded in it. They should initialise it by calling::
void fscache_operation_init(struct fscache_operation *op,
fscache_operation_release_t release);
with the operation to be initialised and the release function to use.
The op->flags parameter should be set to indicate the CPU time provision and
the exclusivity (see the Parameters section).
The op->fast_work, op->slow_work and op->processor flags should be set as
appropriate for the CPU time provision (see the Parameters section).
FSCACHE_OP_WAITING may be set in op->flags prior to each submission of the
operation and waited for afterwards.
Parameters
==========
There are a number of parameters that can be set in the operation record's flag
parameter. There are three options for the provision of CPU time in these
operations:
(1) The operation may be done synchronously (FSCACHE_OP_MYTHREAD). A thread
may decide it wants to handle an operation itself without deferring it to
another thread.
This is, for example, used in read operations for calling readpages() on
the backing filesystem in CacheFiles. Although readpages() does an
asynchronous data fetch, the determination of whether pages exist is done
synchronously - and the netfs does not proceed until this has been
determined.
If this option is to be used, FSCACHE_OP_WAITING must be set in op->flags
before submitting the operation, and the operating thread must wait for it
to be cleared before proceeding::
wait_on_bit(&op->flags, FSCACHE_OP_WAITING,
TASK_UNINTERRUPTIBLE);
(2) The operation may be fast asynchronous (FSCACHE_OP_FAST), in which case it
will be given to keventd to process. Such an operation is not permitted
to sleep on I/O.
This is, for example, used by CacheFiles to copy data from a backing fs
page to a netfs page after the backing fs has read the page in.
If this option is used, op->fast_work and op->processor must be
initialised before submitting the operation::
INIT_WORK(&op->fast_work, do_some_work);
(3) The operation may be slow asynchronous (FSCACHE_OP_SLOW), in which case it
will be given to the slow work facility to process. Such an operation is
permitted to sleep on I/O.
This is, for example, used by FS-Cache to handle background writes of
pages that have just been fetched from a remote server.
If this option is used, op->slow_work and op->processor must be
initialised before submitting the operation::
fscache_operation_init_slow(op, processor)
Furthermore, operations may be one of two types:
(1) Exclusive (FSCACHE_OP_EXCLUSIVE). Operations of this type may not run in
conjunction with any other operation on the object being operated upon.
An example of this is the attribute change operation, in which the file
being written to may need truncation.
(2) Shareable. Operations of this type may be running simultaneously. It's
up to the operation implementation to prevent interference between other
operations running at the same time.
Procedure
=========
Operations are used through the following procedure:
(1) The submitting thread must allocate the operation and initialise it
itself. Normally this would be part of a more specific structure with the
generic op embedded within.
(2) The submitting thread must then submit the operation for processing using
one of the following two functions::
int fscache_submit_op(struct fscache_object *object,
struct fscache_operation *op);
int fscache_submit_exclusive_op(struct fscache_object *object,
struct fscache_operation *op);
The first function should be used to submit non-exclusive ops and the
second to submit exclusive ones. The caller must still set the
FSCACHE_OP_EXCLUSIVE flag.
If successful, both functions will assign the operation to the specified
object and return 0. -ENOBUFS will be returned if the object specified is
permanently unavailable.
The operation manager will defer operations on an object that is still
undergoing lookup or creation. The operation will also be deferred if an
operation of conflicting exclusivity is in progress on the object.
If the operation is asynchronous, the manager will retain a reference to
it, so the caller should put their reference to it by passing it to::
void fscache_put_operation(struct fscache_operation *op);
(3) If the submitting thread wants to do the work itself, and has marked the
operation with FSCACHE_OP_MYTHREAD, then it should monitor
FSCACHE_OP_WAITING as described above and check the state of the object if
necessary (the object might have died while the thread was waiting).
When it has finished doing its processing, it should call
fscache_op_complete() and fscache_put_operation() on it.
(4) The operation holds an effective lock upon the object, preventing other
exclusive ops conflicting until it is released. The operation can be
enqueued for further immediate asynchronous processing by adjusting the
CPU time provisioning option if necessary, eg::
op->flags &= ~FSCACHE_OP_TYPE;
op->flags |= ~FSCACHE_OP_FAST;
and calling::
void fscache_enqueue_operation(struct fscache_operation *op)
This can be used to allow other things to have use of the worker thread
pools.
Asynchronous Callback
=====================
When used in asynchronous mode, the worker thread pool will invoke the
processor method with a pointer to the operation. This should then get at the
container struct by using container_of()::
static void fscache_write_op(struct fscache_operation *_op)
{
struct fscache_storage *op =
container_of(_op, struct fscache_storage, op);
...
}
The caller holds a reference on the operation, and will invoke
fscache_put_operation() when the processor function returns. The processor
function is at liberty to call fscache_enqueue_operation() or to take extra
references.

16
Documentation/filesystems/netfs_library.rst
View File

@ -454,7 +454,8 @@ operation table looks like the following::
void *term_func_priv);
int (*prepare_write)(struct netfs_cache_resources *cres,
loff_t *_start, size_t *_len, loff_t i_size);
loff_t *_start, size_t *_len, loff_t i_size,
bool no_space_allocated_yet);
int (*write)(struct netfs_cache_resources *cres,
loff_t start_pos,
@ -515,11 +516,14 @@ The methods defined in the table are:
* ``prepare_write()``
[Required] Called to adjust a write to the cache and check that there is
sufficient space in the cache. The start and length values indicate the
size of the write that netfslib is proposing, and this can be adjusted by
the cache to respect DIO boundaries. The file size is passed for
information.
[Required] Called to prepare a write to the cache to take place. This
involves checking to see whether the cache has sufficient space to honour
the write. ``*_start`` and ``*_len`` indicate the region to be written; the
region can be shrunk or it can be expanded to a page boundary either way as
necessary to align for direct I/O. i_size holds the size of the object and
is provided for reference. no_space_allocated_yet is set to true if the
caller is certain that no data has been written to that region - for example
if it tried to do a read from there already.
* ``write()``

193
fs/9p/cache.c
View File

@ -16,186 +16,61 @@
#include "v9fs.h"
#include "cache.h"
#define CACHETAG_LEN 11
struct fscache_netfs v9fs_cache_netfs = {
.name = "9p",
.version = 0,
};
/*
* v9fs_random_cachetag - Generate a random tag to be associated
* with a new cache session.
*
* The value of jiffies is used for a fairly randomly cache tag.
*/
static
int v9fs_random_cachetag(struct v9fs_session_info *v9ses)
int v9fs_cache_session_get_cookie(struct v9fs_session_info *v9ses,
const char *dev_name)
{
v9ses->cachetag = kmalloc(CACHETAG_LEN, GFP_KERNEL);
if (!v9ses->cachetag)
struct fscache_volume *vcookie;
char *name, *p;
name = kasprintf(GFP_KERNEL, "9p,%s,%s",
dev_name, v9ses->cachetag ?: v9ses->aname);
if (!name)
return -ENOMEM;
return scnprintf(v9ses->cachetag, CACHETAG_LEN, "%lu", jiffies);
}
for (p = name; *p; p++)
if (*p == '/')
*p = ';';
const struct fscache_cookie_def v9fs_cache_session_index_def = {
.name = "9P.session",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
void v9fs_cache_session_get_cookie(struct v9fs_session_info *v9ses)
{
/* If no cache session tag was specified, we generate a random one. */
if (!v9ses->cachetag) {
if (v9fs_random_cachetag(v9ses) < 0) {
v9ses->fscache = NULL;
kfree(v9ses->cachetag);
v9ses->cachetag = NULL;
return;
vcookie = fscache_acquire_volume(name, NULL, NULL, 0);
p9_debug(P9_DEBUG_FSC, "session %p get volume %p (%s)\n",
v9ses, vcookie, name);
if (IS_ERR(vcookie)) {
if (vcookie != ERR_PTR(-EBUSY)) {
kfree(name);
return PTR_ERR(vcookie);
}
pr_err("Cache volume key already in use (%s)\n", name);
vcookie = NULL;
}
v9ses->fscache = fscache_acquire_cookie(v9fs_cache_netfs.primary_index,
&v9fs_cache_session_index_def,
v9ses->cachetag,
strlen(v9ses->cachetag),
NULL, 0,
v9ses, 0, true);
p9_debug(P9_DEBUG_FSC, "session %p get cookie %p\n",
v9ses, v9ses->fscache);
v9ses->fscache = vcookie;
kfree(name);
return 0;
}
void v9fs_cache_session_put_cookie(struct v9fs_session_info *v9ses)
{
p9_debug(P9_DEBUG_FSC, "session %p put cookie %p\n",
v9ses, v9ses->fscache);
fscache_relinquish_cookie(v9ses->fscache, NULL, false);
v9ses->fscache = NULL;
}
static enum
fscache_checkaux v9fs_cache_inode_check_aux(void *cookie_netfs_data,
const void *buffer,
uint16_t buflen,
loff_t object_size)
{
const struct v9fs_inode *v9inode = cookie_netfs_data;
if (buflen != sizeof(v9inode->qid.version))
return FSCACHE_CHECKAUX_OBSOLETE;
if (memcmp(buffer, &v9inode->qid.version,
sizeof(v9inode->qid.version)))
return FSCACHE_CHECKAUX_OBSOLETE;
return FSCACHE_CHECKAUX_OKAY;
}
const struct fscache_cookie_def v9fs_cache_inode_index_def = {
.name = "9p.inode",
.type = FSCACHE_COOKIE_TYPE_DATAFILE,
.check_aux = v9fs_cache_inode_check_aux,
};
void v9fs_cache_inode_get_cookie(struct inode *inode)
{
struct v9fs_inode *v9inode;
struct v9fs_session_info *v9ses;
__le32 version;
__le64 path;
if (!S_ISREG(inode->i_mode))
return;
v9inode = V9FS_I(inode);
if (v9inode->fscache)
if (WARN_ON(v9inode->fscache))
return;
version = cpu_to_le32(v9inode->qid.version);
path = cpu_to_le64(v9inode->qid.path);
v9ses = v9fs_inode2v9ses(inode);
v9inode->fscache = fscache_acquire_cookie(v9ses->fscache,
&v9fs_cache_inode_index_def,
&v9inode->qid.path,
sizeof(v9inode->qid.path),
&v9inode->qid.version,
sizeof(v9inode->qid.version),
v9inode,
i_size_read(&v9inode->vfs_inode),
true);
v9inode->fscache =
fscache_acquire_cookie(v9fs_session_cache(v9ses),
0,
&path, sizeof(path),
&version, sizeof(version),
i_size_read(&v9inode->vfs_inode));
p9_debug(P9_DEBUG_FSC, "inode %p get cookie %p\n",
inode, v9inode->fscache);
}
void v9fs_cache_inode_put_cookie(struct inode *inode)
{
struct v9fs_inode *v9inode = V9FS_I(inode);
if (!v9inode->fscache)
return;
p9_debug(P9_DEBUG_FSC, "inode %p put cookie %p\n",
inode, v9inode->fscache);
fscache_relinquish_cookie(v9inode->fscache, &v9inode->qid.version,
false);
v9inode->fscache = NULL;
}
void v9fs_cache_inode_flush_cookie(struct inode *inode)
{
struct v9fs_inode *v9inode = V9FS_I(inode);
if (!v9inode->fscache)
return;
p9_debug(P9_DEBUG_FSC, "inode %p flush cookie %p\n",
inode, v9inode->fscache);
fscache_relinquish_cookie(v9inode->fscache, NULL, true);
v9inode->fscache = NULL;
}
void v9fs_cache_inode_set_cookie(struct inode *inode, struct file *filp)
{
struct v9fs_inode *v9inode = V9FS_I(inode);
if (!v9inode->fscache)
return;
mutex_lock(&v9inode->fscache_lock);
if ((filp->f_flags & O_ACCMODE) != O_RDONLY)
v9fs_cache_inode_flush_cookie(inode);
else
v9fs_cache_inode_get_cookie(inode);
mutex_unlock(&v9inode->fscache_lock);
}
void v9fs_cache_inode_reset_cookie(struct inode *inode)
{
struct v9fs_inode *v9inode = V9FS_I(inode);
struct v9fs_session_info *v9ses;
struct fscache_cookie *old;
if (!v9inode->fscache)
return;
old = v9inode->fscache;
mutex_lock(&v9inode->fscache_lock);
fscache_relinquish_cookie(v9inode->fscache, NULL, true);
v9ses = v9fs_inode2v9ses(inode);
v9inode->fscache = fscache_acquire_cookie(v9ses->fscache,
&v9fs_cache_inode_index_def,
&v9inode->qid.path,
sizeof(v9inode->qid.path),
&v9inode->qid.version,
sizeof(v9inode->qid.version),
v9inode,
i_size_read(&v9inode->vfs_inode),
true);
p9_debug(P9_DEBUG_FSC, "inode %p revalidating cookie old %p new %p\n",
inode, old, v9inode->fscache);
mutex_unlock(&v9inode->fscache_lock);
}

25
fs/9p/cache.h
View File

@ -7,26 +7,15 @@
#ifndef _9P_CACHE_H
#define _9P_CACHE_H
#define FSCACHE_USE_NEW_IO_API
#include <linux/fscache.h>
#ifdef CONFIG_9P_FSCACHE
extern struct fscache_netfs v9fs_cache_netfs;
extern const struct fscache_cookie_def v9fs_cache_session_index_def;
extern const struct fscache_cookie_def v9fs_cache_inode_index_def;
extern void v9fs_cache_session_get_cookie(struct v9fs_session_info *v9ses);
extern void v9fs_cache_session_put_cookie(struct v9fs_session_info *v9ses);
extern int v9fs_cache_session_get_cookie(struct v9fs_session_info *v9ses,
const char *dev_name);
extern void v9fs_cache_inode_get_cookie(struct inode *inode);
extern void v9fs_cache_inode_put_cookie(struct inode *inode);
extern void v9fs_cache_inode_flush_cookie(struct inode *inode);
extern void v9fs_cache_inode_set_cookie(struct inode *inode, struct file *filp);
extern void v9fs_cache_inode_reset_cookie(struct inode *inode);
extern int __v9fs_cache_register(void);
extern void __v9fs_cache_unregister(void);
#else /* CONFIG_9P_FSCACHE */
@ -34,13 +23,5 @@ static inline void v9fs_cache_inode_get_cookie(struct inode *inode)
{
}
static inline void v9fs_cache_inode_put_cookie(struct inode *inode)
{
}
static inline void v9fs_cache_inode_set_cookie(struct inode *inode, struct file *file)
{
}
#endif /* CONFIG_9P_FSCACHE */
#endif /* _9P_CACHE_H */

17
fs/9p/v9fs.c
View File

@ -469,7 +469,11 @@ struct p9_fid *v9fs_session_init(struct v9fs_session_info *v9ses,
#ifdef CONFIG_9P_FSCACHE
/* register the session for caching */
v9fs_cache_session_get_cookie(v9ses);
if (v9ses->cache == CACHE_LOOSE || v9ses->cache == CACHE_FSCACHE) {
rc = v9fs_cache_session_get_cookie(v9ses, dev_name);
if (rc < 0)
goto err_clnt;
}
#endif
spin_lock(&v9fs_sessionlist_lock);
list_add(&v9ses->slist, &v9fs_sessionlist);
@ -502,8 +506,7 @@ void v9fs_session_close(struct v9fs_session_info *v9ses)
}
#ifdef CONFIG_9P_FSCACHE
if (v9ses->fscache)
v9fs_cache_session_put_cookie(v9ses);
fscache_relinquish_volume(v9fs_session_cache(v9ses), NULL, false);
kfree(v9ses->cachetag);
#endif
kfree(v9ses->uname);
@ -665,20 +668,12 @@ static int v9fs_cache_register(void)
ret = v9fs_init_inode_cache();
if (ret < 0)
return ret;
#ifdef CONFIG_9P_FSCACHE
ret = fscache_register_netfs(&v9fs_cache_netfs);
if (ret < 0)
v9fs_destroy_inode_cache();
#endif
return ret;
}
static void v9fs_cache_unregister(void)
{
v9fs_destroy_inode_cache();
#ifdef CONFIG_9P_FSCACHE
fscache_unregister_netfs(&v9fs_cache_netfs);
#endif
}
/**

13
fs/9p/v9fs.h
View File

@ -89,7 +89,7 @@ struct v9fs_session_info {
unsigned int cache;
#ifdef CONFIG_9P_FSCACHE
char *cachetag;
struct fscache_cookie *fscache;
struct fscache_volume *fscache;
#endif
char *uname; /* user name to mount as */
@ -109,7 +109,6 @@ struct v9fs_session_info {
struct v9fs_inode {
#ifdef CONFIG_9P_FSCACHE
struct mutex fscache_lock;
struct fscache_cookie *fscache;
#endif
struct p9_qid qid;
@ -133,6 +132,16 @@ static inline struct fscache_cookie *v9fs_inode_cookie(struct v9fs_inode *v9inod
#endif
}
static inline struct fscache_volume *v9fs_session_cache(struct v9fs_session_info *v9ses)
{
#ifdef CONFIG_9P_FSCACHE
return v9ses->fscache;
#else
return NULL;
#endif
}
extern int v9fs_show_options(struct seq_file *m, struct dentry *root);
struct p9_fid *v9fs_session_init(struct v9fs_session_info *v9ses,

57
fs/9p/vfs_addr.c
View File

@ -16,6 +16,7 @@
#include <linux/pagemap.h>
#include <linux/idr.h>
#include <linux/sched.h>
#include <linux/swap.h>
#include <linux/uio.h>
#include <linux/netfs.h>
#include <net/9p/9p.h>
@ -78,7 +79,7 @@ static bool v9fs_is_cache_enabled(struct inode *inode)
{
struct fscache_cookie *cookie = v9fs_inode_cookie(V9FS_I(inode));
return fscache_cookie_enabled(cookie) && !hlist_empty(&cookie->backing_objects);
return fscache_cookie_enabled(cookie) && cookie->cache_priv;
}
/**
@ -87,9 +88,13 @@ static bool v9fs_is_cache_enabled(struct inode *inode)
*/
static int v9fs_begin_cache_operation(struct netfs_read_request *rreq)
{
#ifdef CONFIG_9P_FSCACHE
struct fscache_cookie *cookie = v9fs_inode_cookie(V9FS_I(rreq->inode));
return fscache_begin_read_operation(rreq, cookie);
return fscache_begin_read_operation(&rreq->cache_resources, cookie);
#else
return -ENOBUFS;
#endif
}
static const struct netfs_read_request_ops v9fs_req_ops = {
@ -133,16 +138,18 @@ static void v9fs_vfs_readahead(struct readahead_control *ractl)
static int v9fs_release_page(struct page *page, gfp_t gfp)
{
struct folio *folio = page_folio(page);
struct inode *inode = folio_inode(folio);
if (folio_test_private(folio))
return 0;
#ifdef CONFIG_9P_FSCACHE
if (folio_test_fscache(folio)) {
if (!(gfp & __GFP_DIRECT_RECLAIM) || !(gfp & __GFP_FS))
if (current_is_kswapd() || !(gfp & __GFP_FS))
return 0;
folio_wait_fscache(folio);
}
#endif
fscache_note_page_release(v9fs_inode_cookie(V9FS_I(inode)));
return 1;
}
@ -161,10 +168,25 @@ static void v9fs_invalidate_page(struct page *page, unsigned int offset,
folio_wait_fscache(folio);
}
static void v9fs_write_to_cache_done(void *priv, ssize_t transferred_or_error,
bool was_async)
{
struct v9fs_inode *v9inode = priv;
__le32 version;
if (IS_ERR_VALUE(transferred_or_error) &&
transferred_or_error != -ENOBUFS) {
version = cpu_to_le32(v9inode->qid.version);
fscache_invalidate(v9fs_inode_cookie(v9inode), &version,
i_size_read(&v9inode->vfs_inode), 0);
}
}
static int v9fs_vfs_write_folio_locked(struct folio *folio)
{
struct inode *inode = folio_inode(folio);
struct v9fs_inode *v9inode = V9FS_I(inode);
struct fscache_cookie *cookie = v9fs_inode_cookie(v9inode);
loff_t start = folio_pos(folio);
loff_t i_size = i_size_read(inode);
struct iov_iter from;
@ -181,10 +203,21 @@ static int v9fs_vfs_write_folio_locked(struct folio *folio)
/* We should have writeback_fid always set */
BUG_ON(!v9inode->writeback_fid);
folio_wait_fscache(folio);
folio_start_writeback(folio);
p9_client_write(v9inode->writeback_fid, start, &from, &err);
if (err == 0 &&
fscache_cookie_enabled(cookie) &&
test_bit(FSCACHE_COOKIE_IS_CACHING, &cookie->flags)) {
folio_start_fscache(folio);
fscache_write_to_cache(v9fs_inode_cookie(v9inode),
folio_mapping(folio), start, len, i_size,
v9fs_write_to_cache_done, v9inode,
true);
}
folio_end_writeback(folio);
return err;
}
@ -303,6 +336,7 @@ static int v9fs_write_end(struct file *filp, struct address_space *mapping,
loff_t last_pos = pos + copied;
struct folio *folio = page_folio(subpage);
struct inode *inode = mapping->host;
struct v9fs_inode *v9inode = V9FS_I(inode);
p9_debug(P9_DEBUG_VFS, "filp %p, mapping %p\n", filp, mapping);
@ -322,6 +356,7 @@ static int v9fs_write_end(struct file *filp, struct address_space *mapping,
if (last_pos > inode->i_size) {
inode_add_bytes(inode, last_pos - inode->i_size);
i_size_write(inode, last_pos);
fscache_update_cookie(v9fs_inode_cookie(v9inode), NULL, &last_pos);
}
folio_mark_dirty(folio);
out:
@ -331,11 +366,25 @@ static int v9fs_write_end(struct file *filp, struct address_space *mapping,
return copied;
}
#ifdef CONFIG_9P_FSCACHE
/*
* Mark a page as having been made dirty and thus needing writeback. We also
* need to pin the cache object to write back to.
*/
static int v9fs_set_page_dirty(struct page *page)
{
struct v9fs_inode *v9inode = V9FS_I(page->mapping->host);
return fscache_set_page_dirty(page, v9fs_inode_cookie(v9inode));
}
#else
#define v9fs_set_page_dirty __set_page_dirty_nobuffers
#endif
const struct address_space_operations v9fs_addr_operations = {
.readpage = v9fs_vfs_readpage,
.readahead = v9fs_vfs_readahead,
.set_page_dirty = __set_page_dirty_nobuffers,
.set_page_dirty = v9fs_set_page_dirty,
.writepage = v9fs_vfs_writepage,
.write_begin = v9fs_write_begin,
.write_end = v9fs_write_end,

13
fs/9p/vfs_dir.c
View File

@ -17,6 +17,7 @@
#include <linux/idr.h>
#include <linux/slab.h>
#include <linux/uio.h>
#include <linux/fscache.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
@ -205,7 +206,10 @@ static int v9fs_dir_readdir_dotl(struct file *file, struct dir_context *ctx)
int v9fs_dir_release(struct inode *inode, struct file *filp)
{
struct v9fs_inode *v9inode = V9FS_I(inode);
struct p9_fid *fid;
__le32 version;
loff_t i_size;
fid = filp->private_data;
p9_debug(P9_DEBUG_VFS, "inode: %p filp: %p fid: %d\n",
@ -216,6 +220,15 @@ int v9fs_dir_release(struct inode *inode, struct file *filp)
spin_unlock(&inode->i_lock);
p9_client_clunk(fid);
}
if ((filp->f_mode & FMODE_WRITE)) {
version = cpu_to_le32(v9inode->qid.version);
i_size = i_size_read(inode);
fscache_unuse_cookie(v9fs_inode_cookie(v9inode),
&version, &i_size);
} else {
fscache_unuse_cookie(v9fs_inode_cookie(v9inode), NULL, NULL);
}
return 0;
}

3
fs/9p/vfs_file.c
View File

@ -93,7 +93,8 @@ int v9fs_file_open(struct inode *inode, struct file *file)
}
mutex_unlock(&v9inode->v_mutex);
if (v9ses->cache == CACHE_LOOSE || v9ses->cache == CACHE_FSCACHE)
v9fs_cache_inode_set_cookie(inode, file);
fscache_use_cookie(v9fs_inode_cookie(v9inode),
file->f_mode & FMODE_WRITE);
v9fs_open_fid_add(inode, fid);
return 0;
out_error:

26
fs/9p/vfs_inode.c
View File

@ -233,7 +233,6 @@ struct inode *v9fs_alloc_inode(struct super_block *sb)
return NULL;
#ifdef CONFIG_9P_FSCACHE
v9inode->fscache = NULL;
mutex_init(&v9inode->fscache_lock);
#endif
v9inode->writeback_fid = NULL;
v9inode->cache_validity = 0;
@ -381,12 +380,16 @@ struct inode *v9fs_get_inode(struct super_block *sb, umode_t mode, dev_t rdev)
void v9fs_evict_inode(struct inode *inode)
{
struct v9fs_inode *v9inode = V9FS_I(inode);
__le32 version;
truncate_inode_pages_final(&inode->i_data);
version = cpu_to_le32(v9inode->qid.version);
fscache_clear_inode_writeback(v9fs_inode_cookie(v9inode), inode,
&version);
clear_inode(inode);
filemap_fdatawrite(&inode->i_data);
v9fs_cache_inode_put_cookie(inode);
fscache_relinquish_cookie(v9fs_inode_cookie(v9inode), false);
/* clunk the fid stashed in writeback_fid */
if (v9inode->writeback_fid) {
p9_client_clunk(v9inode->writeback_fid);
@ -869,7 +872,8 @@ v9fs_vfs_atomic_open(struct inode *dir, struct dentry *dentry,
file->private_data = fid;
if (v9ses->cache == CACHE_LOOSE || v9ses->cache == CACHE_FSCACHE)
v9fs_cache_inode_set_cookie(d_inode(dentry), file);
fscache_use_cookie(v9fs_inode_cookie(v9inode),
file->f_mode & FMODE_WRITE);
v9fs_open_fid_add(inode, fid);
file->f_mode |= FMODE_CREATED;
@ -1072,6 +1076,8 @@ static int v9fs_vfs_setattr(struct user_namespace *mnt_userns,
struct dentry *dentry, struct iattr *iattr)
{
int retval, use_dentry = 0;
struct inode *inode = d_inode(dentry);
struct v9fs_inode *v9inode = V9FS_I(inode);
struct v9fs_session_info *v9ses;
struct p9_fid *fid = NULL;
struct p9_wstat wstat;
@ -1117,7 +1123,7 @@ static int v9fs_vfs_setattr(struct user_namespace *mnt_userns,
/* Write all dirty data */
if (d_is_reg(dentry))
filemap_write_and_wait(d_inode(dentry)->i_mapping);
filemap_write_and_wait(inode->i_mapping);
retval = p9_client_wstat(fid, &wstat);
@ -1128,13 +1134,15 @@ static int v9fs_vfs_setattr(struct user_namespace *mnt_userns,
return retval;
if ((iattr->ia_valid & ATTR_SIZE) &&
iattr->ia_size != i_size_read(d_inode(dentry)))
truncate_setsize(d_inode(dentry), iattr->ia_size);
iattr->ia_size != i_size_read(inode)) {
truncate_setsize(inode, iattr->ia_size);
fscache_resize_cookie(v9fs_inode_cookie(v9inode), iattr->ia_size);
}
v9fs_invalidate_inode_attr(d_inode(dentry));
v9fs_invalidate_inode_attr(inode);
setattr_copy(&init_user_ns, d_inode(dentry), iattr);
mark_inode_dirty(d_inode(dentry));
setattr_copy(&init_user_ns, inode, iattr);
mark_inode_dirty(inode);
return 0;
}

3
fs/9p/vfs_inode_dotl.c
View File

@ -344,7 +344,8 @@ v9fs_vfs_atomic_open_dotl(struct inode *dir, struct dentry *dentry,
goto err_clunk_old_fid;
file->private_data = ofid;
if (v9ses->cache == CACHE_LOOSE || v9ses->cache == CACHE_FSCACHE)
v9fs_cache_inode_set_cookie(inode, file);
fscache_use_cookie(v9fs_inode_cookie(v9inode),
file->f_mode & FMODE_WRITE);
v9fs_open_fid_add(inode, ofid);
file->f_mode |= FMODE_CREATED;
out:

3
fs/9p/vfs_super.c
View File

@ -20,6 +20,7 @@
#include <linux/slab.h>
#include <linux/statfs.h>
#include <linux/magic.h>
#include <linux/fscache.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
@ -309,6 +310,7 @@ static int v9fs_write_inode(struct inode *inode,
__mark_inode_dirty(inode, I_DIRTY_DATASYNC);
return ret;
}
fscache_unpin_writeback(wbc, v9fs_inode_cookie(v9inode));
return 0;
}
@ -332,6 +334,7 @@ static int v9fs_write_inode_dotl(struct inode *inode,
__mark_inode_dirty(inode, I_DIRTY_DATASYNC);
return ret;
}
fscache_unpin_writeback(wbc, v9fs_inode_cookie(v9inode));
return 0;
}

3
fs/afs/Makefile
View File

@ -3,10 +3,7 @@
# Makefile for Red Hat Linux AFS client.
#
afs-cache-$(CONFIG_AFS_FSCACHE) := cache.o
kafs-y := \
$(afs-cache-y) \
addr_list.o \
callback.o \
cell.o \

68
fs/afs/cache.c
View File

@ -1,68 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* AFS caching stuff
*
* Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/sched.h>
#include "internal.h"
static enum fscache_checkaux afs_vnode_cache_check_aux(void *cookie_netfs_data,
const void *buffer,
uint16_t buflen,
loff_t object_size);
struct fscache_netfs afs_cache_netfs = {
.name = "afs",
.version = 2,
};
struct fscache_cookie_def afs_cell_cache_index_def = {
.name = "AFS.cell",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
struct fscache_cookie_def afs_volume_cache_index_def = {
.name = "AFS.volume",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
struct fscache_cookie_def afs_vnode_cache_index_def = {
.name = "AFS.vnode",
.type = FSCACHE_COOKIE_TYPE_DATAFILE,
.check_aux = afs_vnode_cache_check_aux,
};
/*
* check that the auxiliary data indicates that the entry is still valid
*/
static enum fscache_checkaux afs_vnode_cache_check_aux(void *cookie_netfs_data,
const void *buffer,
uint16_t buflen,
loff_t object_size)
{
struct afs_vnode *vnode = cookie_netfs_data;
struct afs_vnode_cache_aux aux;
_enter("{%llx,%x,%llx},%p,%u",
vnode->fid.vnode, vnode->fid.unique, vnode->status.data_version,
buffer, buflen);
memcpy(&aux, buffer, sizeof(aux));
/* check the size of the data is what we're expecting */
if (buflen != sizeof(aux)) {
_leave(" = OBSOLETE [len %hx != %zx]", buflen, sizeof(aux));
return FSCACHE_CHECKAUX_OBSOLETE;
}
if (vnode->status.data_version != aux.data_version) {
_leave(" = OBSOLETE [vers %llx != %llx]",
aux.data_version, vnode->status.data_version);
return FSCACHE_CHECKAUX_OBSOLETE;
}
_leave(" = SUCCESS");
return FSCACHE_CHECKAUX_OKAY;
}

12
fs/afs/cell.c
View File

@ -680,13 +680,6 @@ static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
return ret;
}
#ifdef CONFIG_AFS_FSCACHE
cell->cache = fscache_acquire_cookie(afs_cache_netfs.primary_index,
&afs_cell_cache_index_def,
cell->name, strlen(cell->name),
NULL, 0,
cell, 0, true);
#endif
ret = afs_proc_cell_setup(cell);
if (ret < 0)
return ret;
@ -723,11 +716,6 @@ static void afs_deactivate_cell(struct afs_net *net, struct afs_cell *cell)
afs_dynroot_rmdir(net, cell);
mutex_unlock(&net->proc_cells_lock);
#ifdef CONFIG_AFS_FSCACHE
fscache_relinquish_cookie(cell->cache, NULL, false);
cell->cache = NULL;
#endif
_leave("");
}

38
fs/afs/file.c
View File

@ -14,6 +14,7 @@
#include <linux/gfp.h>
#include <linux/task_io_accounting_ops.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/netfs.h>
#include "internal.h"
@ -158,7 +159,9 @@ int afs_open(struct inode *inode, struct file *file)
if (file->f_flags & O_TRUNC)
set_bit(AFS_VNODE_NEW_CONTENT, &vnode->flags);
fscache_use_cookie(afs_vnode_cache(vnode), file->f_mode & FMODE_WRITE);
file->private_data = af;
_leave(" = 0");
return 0;
@ -177,8 +180,10 @@ int afs_open(struct inode *inode, struct file *file)
*/
int afs_release(struct inode *inode, struct file *file)
{
struct afs_vnode_cache_aux aux;
struct afs_vnode *vnode = AFS_FS_I(inode);
struct afs_file *af = file->private_data;
loff_t i_size;
int ret = 0;
_enter("{%llx:%llu},", vnode->fid.vid, vnode->fid.vnode);
@ -189,6 +194,15 @@ int afs_release(struct inode *inode, struct file *file)
file->private_data = NULL;
if (af->wb)
afs_put_wb_key(af->wb);
if ((file->f_mode & FMODE_WRITE)) {
i_size = i_size_read(&vnode->vfs_inode);
afs_set_cache_aux(vnode, &aux);
fscache_unuse_cookie(afs_vnode_cache(vnode), &aux, &i_size);
} else {
fscache_unuse_cookie(afs_vnode_cache(vnode), NULL, NULL);
}
key_put(af->key);
kfree(af);
afs_prune_wb_keys(vnode);
@ -354,14 +368,19 @@ static bool afs_is_cache_enabled(struct inode *inode)
{
struct fscache_cookie *cookie = afs_vnode_cache(AFS_FS_I(inode));
return fscache_cookie_enabled(cookie) && !hlist_empty(&cookie->backing_objects);
return fscache_cookie_enabled(cookie) && cookie->cache_priv;
}
static int afs_begin_cache_operation(struct netfs_read_request *rreq)
{
#ifdef CONFIG_AFS_FSCACHE
struct afs_vnode *vnode = AFS_FS_I(rreq->inode);
return fscache_begin_read_operation(rreq, afs_vnode_cache(vnode));
return fscache_begin_read_operation(&rreq->cache_resources,
afs_vnode_cache(vnode));
#else
return -ENOBUFS;
#endif
}
static int afs_check_write_begin(struct file *file, loff_t pos, unsigned len,
@ -398,6 +417,12 @@ static void afs_readahead(struct readahead_control *ractl)
netfs_readahead(ractl, &afs_req_ops, NULL);
}
int afs_write_inode(struct inode *inode, struct writeback_control *wbc)
{
fscache_unpin_writeback(wbc, afs_vnode_cache(AFS_FS_I(inode)));
return 0;
}
/*
* Adjust the dirty region of the page on truncation or full invalidation,
* getting rid of the markers altogether if the region is entirely invalidated.
@ -480,23 +505,24 @@ static void afs_invalidatepage(struct page *page, unsigned int offset,
* release a page and clean up its private state if it's not busy
* - return true if the page can now be released, false if not
*/
static int afs_releasepage(struct page *page, gfp_t gfp_flags)
static int afs_releasepage(struct page *page, gfp_t gfp)
{
struct folio *folio = page_folio(page);
struct afs_vnode *vnode = AFS_FS_I(folio_inode(folio));
_enter("{{%llx:%llu}[%lu],%lx},%x",
vnode->fid.vid, vnode->fid.vnode, folio_index(folio), folio->flags,
gfp_flags);
gfp);
/* deny if page is being written to the cache and the caller hasn't
* elected to wait */
#ifdef CONFIG_AFS_FSCACHE
if (folio_test_fscache(folio)) {
if (!(gfp_flags & __GFP_DIRECT_RECLAIM) || !(gfp_flags & __GFP_FS))
if (current_is_kswapd() || !(gfp & __GFP_FS))
return false;
folio_wait_fscache(folio);
}
fscache_note_page_release(afs_vnode_cache(vnode));
#endif
if (folio_test_private(folio)) {

99
fs/afs/inode.c
View File

@ -413,9 +413,9 @@ static void afs_get_inode_cache(struct afs_vnode *vnode)
{
#ifdef CONFIG_AFS_FSCACHE
struct {
u32 vnode_id;
u32 unique;
u32 vnode_id_ext[2]; /* Allow for a 96-bit key */
__be32 vnode_id;
__be32 unique;
__be32 vnode_id_ext[2]; /* Allow for a 96-bit key */
} __packed key;
struct afs_vnode_cache_aux aux;
@ -424,17 +424,18 @@ static void afs_get_inode_cache(struct afs_vnode *vnode)
return;
}
key.vnode_id = vnode->fid.vnode;
key.unique = vnode->fid.unique;
key.vnode_id_ext[0] = vnode->fid.vnode >> 32;
key.vnode_id_ext[1] = vnode->fid.vnode_hi;
aux.data_version = vnode->status.data_version;
key.vnode_id = htonl(vnode->fid.vnode);
key.unique = htonl(vnode->fid.unique);
key.vnode_id_ext[0] = htonl(vnode->fid.vnode >> 32);
key.vnode_id_ext[1] = htonl(vnode->fid.vnode_hi);
afs_set_cache_aux(vnode, &aux);
vnode->cache = fscache_acquire_cookie(vnode->volume->cache,
&afs_vnode_cache_index_def,
&key, sizeof(key),
&aux, sizeof(aux),
vnode, vnode->status.size, true);
vnode->cache = fscache_acquire_cookie(
vnode->volume->cache,
vnode->status.type == AFS_FTYPE_FILE ? 0 : FSCACHE_ADV_SINGLE_CHUNK,
&key, sizeof(key),
&aux, sizeof(aux),
vnode->status.size);
#endif
}
@ -563,9 +564,7 @@ static void afs_zap_data(struct afs_vnode *vnode)
{
_enter("{%llx:%llu}", vnode->fid.vid, vnode->fid.vnode);
#ifdef CONFIG_AFS_FSCACHE
fscache_invalidate(vnode->cache);
#endif
afs_invalidate_cache(vnode, 0);
/* nuke all the non-dirty pages that aren't locked, mapped or being
* written back in a regular file and completely discard the pages in a
@ -762,9 +761,8 @@ int afs_drop_inode(struct inode *inode)
*/
void afs_evict_inode(struct inode *inode)
{
struct afs_vnode *vnode;
vnode = AFS_FS_I(inode);
struct afs_vnode_cache_aux aux;
struct afs_vnode *vnode = AFS_FS_I(inode);
_enter("{%llx:%llu.%d}",
vnode->fid.vid,
@ -776,6 +774,9 @@ void afs_evict_inode(struct inode *inode)
ASSERTCMP(inode->i_ino, ==, vnode->fid.vnode);
truncate_inode_pages_final(&inode->i_data);
afs_set_cache_aux(vnode, &aux);
fscache_clear_inode_writeback(afs_vnode_cache(vnode), inode, &aux);
clear_inode(inode);
while (!list_empty(&vnode->wb_keys)) {
@ -786,14 +787,9 @@ void afs_evict_inode(struct inode *inode)
}
#ifdef CONFIG_AFS_FSCACHE
{
struct afs_vnode_cache_aux aux;
aux.data_version = vnode->status.data_version;
fscache_relinquish_cookie(vnode->cache, &aux,
test_bit(AFS_VNODE_DELETED, &vnode->flags));
vnode->cache = NULL;
}
fscache_relinquish_cookie(vnode->cache,
test_bit(AFS_VNODE_DELETED, &vnode->flags));
vnode->cache = NULL;
#endif
afs_prune_wb_keys(vnode);
@ -833,6 +829,9 @@ static void afs_setattr_edit_file(struct afs_operation *op)
if (size < i_size)
truncate_pagecache(inode, size);
if (size != i_size)
fscache_resize_cookie(afs_vnode_cache(vp->vnode),
vp->scb.status.size);
}
}
@ -849,40 +848,67 @@ static const struct afs_operation_ops afs_setattr_operation = {
int afs_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
struct iattr *attr)
{
const unsigned int supported =
ATTR_SIZE | ATTR_MODE | ATTR_UID | ATTR_GID |
ATTR_MTIME | ATTR_MTIME_SET | ATTR_TIMES_SET | ATTR_TOUCH;
struct afs_operation *op;
struct afs_vnode *vnode = AFS_FS_I(d_inode(dentry));
struct inode *inode = &vnode->vfs_inode;
loff_t i_size;
int ret;
_enter("{%llx:%llu},{n=%pd},%x",
vnode->fid.vid, vnode->fid.vnode, dentry,
attr->ia_valid);
if (!(attr->ia_valid & (ATTR_SIZE | ATTR_MODE | ATTR_UID | ATTR_GID |
ATTR_MTIME | ATTR_MTIME_SET | ATTR_TIMES_SET |
ATTR_TOUCH))) {
if (!(attr->ia_valid & supported)) {
_leave(" = 0 [unsupported]");
return 0;
}
i_size = i_size_read(inode);
if (attr->ia_valid & ATTR_SIZE) {
if (!S_ISREG(vnode->vfs_inode.i_mode))
if (!S_ISREG(inode->i_mode))
return -EISDIR;
ret = inode_newsize_ok(&vnode->vfs_inode, attr->ia_size);
ret = inode_newsize_ok(inode, attr->ia_size);
if (ret)
return ret;
if (attr->ia_size == i_size_read(&vnode->vfs_inode))
if (attr->ia_size == i_size)
attr->ia_valid &= ~ATTR_SIZE;
}
/* flush any dirty data outstanding on a regular file */
if (S_ISREG(vnode->vfs_inode.i_mode))
filemap_write_and_wait(vnode->vfs_inode.i_mapping);
fscache_use_cookie(afs_vnode_cache(vnode), true);
/* Prevent any new writebacks from starting whilst we do this. */
down_write(&vnode->validate_lock);
if ((attr->ia_valid & ATTR_SIZE) && S_ISREG(inode->i_mode)) {
loff_t size = attr->ia_size;
/* Wait for any outstanding writes to the server to complete */
loff_t from = min(size, i_size);
loff_t to = max(size, i_size);
ret = filemap_fdatawait_range(inode->i_mapping, from, to);
if (ret < 0)
goto out_unlock;
/* Don't talk to the server if we're just shortening in-memory
* writes that haven't gone to the server yet.
*/
if (!(attr->ia_valid & (supported & ~ATTR_SIZE & ~ATTR_MTIME)) &&
attr->ia_size < i_size &&
attr->ia_size > vnode->status.size) {
truncate_pagecache(inode, attr->ia_size);
fscache_resize_cookie(afs_vnode_cache(vnode),
attr->ia_size);
i_size_write(inode, attr->ia_size);
ret = 0;
goto out_unlock;
}
}
op = afs_alloc_operation(((attr->ia_valid & ATTR_FILE) ?
afs_file_key(attr->ia_file) : NULL),
vnode->volume);
@ -907,6 +933,7 @@ int afs_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
out_unlock:
up_write(&vnode->validate_lock);
fscache_unuse_cookie(afs_vnode_cache(vnode), NULL, NULL);
_leave(" = %d", ret);
return ret;
}

37
fs/afs/internal.h
View File

@ -14,7 +14,6 @@
#include <linux/key.h>
#include <linux/workqueue.h>
#include <linux/sched.h>
#define FSCACHE_USE_NEW_IO_API
#include <linux/fscache.h>
#include <linux/backing-dev.h>
#include <linux/uuid.h>
@ -364,9 +363,6 @@ struct afs_cell {
struct key *anonymous_key; /* anonymous user key for this cell */
struct work_struct manager; /* Manager for init/deinit/dns */
struct hlist_node proc_link; /* /proc cell list link */
#ifdef CONFIG_AFS_FSCACHE
struct fscache_cookie *cache; /* caching cookie */
#endif
time64_t dns_expiry; /* Time AFSDB/SRV record expires */
time64_t last_inactive; /* Time of last drop of usage count */
atomic_t ref; /* Struct refcount */
@ -590,7 +586,7 @@ struct afs_volume {
#define AFS_VOLUME_BUSY 5 /* - T if volume busy notice given */
#define AFS_VOLUME_MAYBE_NO_IBULK 6 /* - T if some servers don't have InlineBulkStatus */
#ifdef CONFIG_AFS_FSCACHE
struct fscache_cookie *cache; /* caching cookie */
struct fscache_volume *cache; /* Caching cookie */
#endif
struct afs_server_list __rcu *servers; /* List of servers on which volume resides */
rwlock_t servers_lock; /* Lock for ->servers */
@ -872,9 +868,24 @@ struct afs_operation {
* Cache auxiliary data.
*/
struct afs_vnode_cache_aux {
u64 data_version;
__be64 data_version;
} __packed;
static inline void afs_set_cache_aux(struct afs_vnode *vnode,
struct afs_vnode_cache_aux *aux)
{
aux->data_version = cpu_to_be64(vnode->status.data_version);
}
static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int flags)
{
struct afs_vnode_cache_aux aux;
afs_set_cache_aux(vnode, &aux);
fscache_invalidate(afs_vnode_cache(vnode), &aux,
i_size_read(&vnode->vfs_inode), flags);
}
/*
* We use folio->private to hold the amount of the folio that we've written to,
* splitting the field into two parts. However, we need to represent a range
@ -962,13 +973,6 @@ extern void afs_merge_fs_addr6(struct afs_addr_list *, __be32 *, u16);
*/
#ifdef CONFIG_AFS_FSCACHE
extern struct fscache_netfs afs_cache_netfs;
extern struct fscache_cookie_def afs_cell_cache_index_def;
extern struct fscache_cookie_def afs_volume_cache_index_def;
extern struct fscache_cookie_def afs_vnode_cache_index_def;
#else
#define afs_cell_cache_index_def (*(struct fscache_cookie_def *) NULL)
#define afs_volume_cache_index_def (*(struct fscache_cookie_def *) NULL)
#define afs_vnode_cache_index_def (*(struct fscache_cookie_def *) NULL)
#endif
/*
@ -1068,6 +1072,7 @@ extern int afs_release(struct inode *, struct file *);
extern int afs_fetch_data(struct afs_vnode *, struct afs_read *);
extern struct afs_read *afs_alloc_read(gfp_t);
extern void afs_put_read(struct afs_read *);
extern int afs_write_inode(struct inode *, struct writeback_control *);
static inline struct afs_read *afs_get_read(struct afs_read *req)
{
@ -1506,7 +1511,7 @@ extern struct afs_vlserver_list *afs_extract_vlserver_list(struct afs_cell *,
* volume.c
*/
extern struct afs_volume *afs_create_volume(struct afs_fs_context *);
extern void afs_activate_volume(struct afs_volume *);
extern int afs_activate_volume(struct afs_volume *);
extern void afs_deactivate_volume(struct afs_volume *);
extern struct afs_volume *afs_get_volume(struct afs_volume *, enum afs_volume_trace);
extern void afs_put_volume(struct afs_net *, struct afs_volume *, enum afs_volume_trace);
@ -1515,7 +1520,11 @@ extern int afs_check_volume_status(struct afs_volume *, struct afs_operation *);
/*
* write.c
*/
#ifdef CONFIG_AFS_FSCACHE
extern int afs_set_page_dirty(struct page *);
#else
#define afs_set_page_dirty __set_page_dirty_nobuffers
#endif
extern int afs_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata);

14
fs/afs/main.c
View File

@ -186,13 +186,6 @@ static int __init afs_init(void)
if (!afs_lock_manager)
goto error_lockmgr;
#ifdef CONFIG_AFS_FSCACHE
/* we want to be able to cache */
ret = fscache_register_netfs(&afs_cache_netfs);
if (ret < 0)
goto error_cache;
#endif
ret = register_pernet_device(&afs_net_ops);
if (ret < 0)
goto error_net;
@ -215,10 +208,6 @@ static int __init afs_init(void)
error_fs:
unregister_pernet_device(&afs_net_ops);
error_net:
#ifdef CONFIG_AFS_FSCACHE
fscache_unregister_netfs(&afs_cache_netfs);
error_cache:
#endif
destroy_workqueue(afs_lock_manager);
error_lockmgr:
destroy_workqueue(afs_async_calls);
@ -245,9 +234,6 @@ static void __exit afs_exit(void)
proc_remove(afs_proc_symlink);
afs_fs_exit();
unregister_pernet_device(&afs_net_ops);
#ifdef CONFIG_AFS_FSCACHE
fscache_unregister_netfs(&afs_cache_netfs);
#endif
destroy_workqueue(afs_lock_manager);
destroy_workqueue(afs_async_calls);
destroy_workqueue(afs_wq);

1
fs/afs/super.c
View File

@ -55,6 +55,7 @@ int afs_net_id;
static const struct super_operations afs_super_ops = {
.statfs = afs_statfs,
.alloc_inode = afs_alloc_inode,
.write_inode = afs_write_inode,
.drop_inode = afs_drop_inode,
.destroy_inode = afs_destroy_inode,
.free_inode = afs_free_inode,

29
fs/afs/volume.c
View File

@ -268,15 +268,30 @@ void afs_put_volume(struct afs_net *net, struct afs_volume *volume,
/*
* Activate a volume.
*/
void afs_activate_volume(struct afs_volume *volume)
int afs_activate_volume(struct afs_volume *volume)
{
#ifdef CONFIG_AFS_FSCACHE
volume->cache = fscache_acquire_cookie(volume->cell->cache,
&afs_volume_cache_index_def,
&volume->vid, sizeof(volume->vid),
NULL, 0,
volume, 0, true);
struct fscache_volume *vcookie;
char *name;
name = kasprintf(GFP_KERNEL, "afs,%s,%llx",
volume->cell->name, volume->vid);
if (!name)
return -ENOMEM;
vcookie = fscache_acquire_volume(name, NULL, NULL, 0);
if (IS_ERR(vcookie)) {
if (vcookie != ERR_PTR(-EBUSY)) {
kfree(name);
return PTR_ERR(vcookie);
}
pr_err("AFS: Cache volume key already in use (%s)\n", name);
vcookie = NULL;
}
volume->cache = vcookie;
kfree(name);
#endif
return 0;
}
/*
@ -287,7 +302,7 @@ void afs_deactivate_volume(struct afs_volume *volume)
_enter("%s", volume->name);
#ifdef CONFIG_AFS_FSCACHE
fscache_relinquish_cookie(volume->cache, NULL,
fscache_relinquish_volume(volume->cache, NULL,
test_bit(AFS_VOLUME_DELETED, &volume->flags));
volume->cache = NULL;
#endif

88
fs/afs/write.c
View File

@ -12,17 +12,30 @@
#include <linux/writeback.h>
#include <linux/pagevec.h>
#include <linux/netfs.h>
#include <linux/fscache.h>
#include "internal.h"
static void afs_write_to_cache(struct afs_vnode *vnode, loff_t start, size_t len,
loff_t i_size, bool caching);
#ifdef CONFIG_AFS_FSCACHE
/*
* mark a page as having been made dirty and thus needing writeback
* Mark a page as having been made dirty and thus needing writeback. We also
* need to pin the cache object to write back to.
*/
int afs_set_page_dirty(struct page *page)
{
_enter("");
return __set_page_dirty_nobuffers(page);
return fscache_set_page_dirty(page, afs_vnode_cache(AFS_FS_I(page->mapping->host)));
}
static void afs_folio_start_fscache(bool caching, struct folio *folio)
{
if (caching)
folio_start_fscache(folio);
}
#else
static void afs_folio_start_fscache(bool caching, struct folio *folio)
{
}
#endif
/*
* prepare to perform part of a write to a page
@ -114,7 +127,7 @@ int afs_write_end(struct file *file, struct address_space *mapping,
unsigned long priv;
unsigned int f, from = offset_in_folio(folio, pos);
unsigned int t, to = from + copied;
loff_t i_size, maybe_i_size;
loff_t i_size, write_end_pos;
_enter("{%llx:%llu},{%lx}",
vnode->fid.vid, vnode->fid.vnode, folio_index(folio));
@ -131,15 +144,16 @@ int afs_write_end(struct file *file, struct address_space *mapping,
if (copied == 0)
goto out;
maybe_i_size = pos + copied;
write_end_pos = pos + copied;
i_size = i_size_read(&vnode->vfs_inode);
if (maybe_i_size > i_size) {
if (write_end_pos > i_size) {
write_seqlock(&vnode->cb_lock);
i_size = i_size_read(&vnode->vfs_inode);
if (maybe_i_size > i_size)
afs_set_i_size(vnode, maybe_i_size);
if (write_end_pos > i_size)
afs_set_i_size(vnode, write_end_pos);
write_sequnlock(&vnode->cb_lock);
fscache_update_cookie(afs_vnode_cache(vnode), NULL, &write_end_pos);
}
if (folio_test_private(folio)) {
@ -418,6 +432,7 @@ static void afs_extend_writeback(struct address_space *mapping,
loff_t start,
loff_t max_len,
bool new_content,
bool caching,
unsigned int *_len)
{
struct pagevec pvec;
@ -464,7 +479,9 @@ static void afs_extend_writeback(struct address_space *mapping,
folio_put(folio);
break;
}
if (!folio_test_dirty(folio) || folio_test_writeback(folio)) {
if (!folio_test_dirty(folio) ||
folio_test_writeback(folio) ||
folio_test_fscache(folio)) {
folio_unlock(folio);
folio_put(folio);
break;
@ -512,6 +529,7 @@ static void afs_extend_writeback(struct address_space *mapping,
BUG();
if (folio_start_writeback(folio))
BUG();
afs_folio_start_fscache(caching, folio);
*_count -= folio_nr_pages(folio);
folio_unlock(folio);
@ -539,6 +557,7 @@ static ssize_t afs_write_back_from_locked_folio(struct address_space *mapping,
unsigned int offset, to, len, max_len;
loff_t i_size = i_size_read(&vnode->vfs_inode);
bool new_content = test_bit(AFS_VNODE_NEW_CONTENT, &vnode->flags);
bool caching = fscache_cookie_enabled(afs_vnode_cache(vnode));
long count = wbc->nr_to_write;
int ret;
@ -546,6 +565,7 @@ static ssize_t afs_write_back_from_locked_folio(struct address_space *mapping,
if (folio_start_writeback(folio))
BUG();
afs_folio_start_fscache(caching, folio);
count -= folio_nr_pages(folio);
@ -572,7 +592,8 @@ static ssize_t afs_write_back_from_locked_folio(struct address_space *mapping,
if (len < max_len &&
(to == folio_size(folio) || new_content))
afs_extend_writeback(mapping, vnode, &count,
start, max_len, new_content, &len);
start, max_len, new_content,
caching, &len);
len = min_t(loff_t, len, max_len);
}
@ -585,12 +606,19 @@ static ssize_t afs_write_back_from_locked_folio(struct address_space *mapping,
if (start < i_size) {
_debug("write back %x @%llx [%llx]", len, start, i_size);
/* Speculatively write to the cache. We have to fix this up
* later if the store fails.
*/
afs_write_to_cache(vnode, start, len, i_size, caching);
iov_iter_xarray(&iter, WRITE, &mapping->i_pages, start, len);
ret = afs_store_data(vnode, &iter, start, false);
} else {
_debug("write discard %x @%llx [%llx]", len, start, i_size);
/* The dirty region was entirely beyond the EOF. */
fscache_clear_page_bits(afs_vnode_cache(vnode),
mapping, start, len, caching);
afs_pages_written_back(vnode, start, len);
ret = 0;
}
@ -649,6 +677,10 @@ int afs_writepage(struct page *subpage, struct writeback_control *wbc)
_enter("{%lx},", folio_index(folio));
#ifdef CONFIG_AFS_FSCACHE
folio_wait_fscache(folio);
#endif
start = folio_index(folio) * PAGE_SIZE;
ret = afs_write_back_from_locked_folio(folio_mapping(folio), wbc,
folio, start, LLONG_MAX - start);
@ -714,10 +746,15 @@ static int afs_writepages_region(struct address_space *mapping,
continue;
}
if (folio_test_writeback(folio)) {
if (folio_test_writeback(folio) ||
folio_test_fscache(folio)) {
folio_unlock(folio);
if (wbc->sync_mode != WB_SYNC_NONE)
if (wbc->sync_mode != WB_SYNC_NONE) {
folio_wait_writeback(folio);
#ifdef CONFIG_AFS_FSCACHE
folio_wait_fscache(folio);
#endif
}
folio_put(folio);
continue;
}
@ -970,3 +1007,28 @@ int afs_launder_page(struct page *subpage)
folio_wait_fscache(folio);
return ret;
}
/*
* Deal with the completion of writing the data to the cache.
*/
static void afs_write_to_cache_done(void *priv, ssize_t transferred_or_error,
bool was_async)
{
struct afs_vnode *vnode = priv;
if (IS_ERR_VALUE(transferred_or_error) &&
transferred_or_error != -ENOBUFS)
afs_invalidate_cache(vnode, 0);
}
/*
* Save the write to the cache also.
*/
static void afs_write_to_cache(struct afs_vnode *vnode,
loff_t start, size_t len, loff_t i_size,
bool caching)
{
fscache_write_to_cache(afs_vnode_cache(vnode),
vnode->vfs_inode.i_mapping, start, len, i_size,
afs_write_to_cache_done, vnode, caching);
}

7
fs/cachefiles/Kconfig
View File

@ -19,3 +19,10 @@ config CACHEFILES_DEBUG
caching on files module. If this is set, the debugging output may be
enabled by setting bits in /sys/modules/cachefiles/parameter/debug or
by including a debugging specifier in /etc/cachefilesd.conf.
config CACHEFILES_ERROR_INJECTION
bool "Provide error injection for cachefiles"
depends on CACHEFILES && SYSCTL
help
This permits error injection to be enabled in cachefiles whilst a
cache is in service.

6
fs/cachefiles/Makefile
View File

@ -4,15 +4,17 @@
#
cachefiles-y := \
bind.o \
cache.o \
daemon.o \
interface.o \
io.o \
key.o \
main.o \
namei.o \
rdwr.o \
security.o \
volume.o \
xattr.o
cachefiles-$(CONFIG_CACHEFILES_ERROR_INJECTION) += error_inject.o
obj-$(CONFIG_CACHEFILES) := cachefiles.o

278
fs/cachefiles/bind.c
View File

@ -1,278 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Bind and unbind a cache from the filesystem backing it
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/module.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/completion.h>
#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/file.h>
#include <linux/namei.h>
#include <linux/mount.h>
#include <linux/statfs.h>
#include <linux/ctype.h>
#include <linux/xattr.h>
#include "internal.h"
static int cachefiles_daemon_add_cache(struct cachefiles_cache *caches);
/*
* bind a directory as a cache
*/
int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
{
_enter("{%u,%u,%u,%u,%u,%u},%s",
cache->frun_percent,
cache->fcull_percent,
cache->fstop_percent,
cache->brun_percent,
cache->bcull_percent,
cache->bstop_percent,
args);
/* start by checking things over */
ASSERT(cache->fstop_percent >= 0 &&
cache->fstop_percent < cache->fcull_percent &&
cache->fcull_percent < cache->frun_percent &&
cache->frun_percent < 100);
ASSERT(cache->bstop_percent >= 0 &&
cache->bstop_percent < cache->bcull_percent &&
cache->bcull_percent < cache->brun_percent &&
cache->brun_percent < 100);
if (*args) {
pr_err("'bind' command doesn't take an argument\n");
return -EINVAL;
}
if (!cache->rootdirname) {
pr_err("No cache directory specified\n");
return -EINVAL;
}
/* don't permit already bound caches to be re-bound */
if (test_bit(CACHEFILES_READY, &cache->flags)) {
pr_err("Cache already bound\n");
return -EBUSY;
}
/* make sure we have copies of the tag and dirname strings */
if (!cache->tag) {
/* the tag string is released by the fops->release()
* function, so we don't release it on error here */
cache->tag = kstrdup("CacheFiles", GFP_KERNEL);
if (!cache->tag)
return -ENOMEM;
}
/* add the cache */
return cachefiles_daemon_add_cache(cache);
}
/*
* add a cache
*/
static int cachefiles_daemon_add_cache(struct cachefiles_cache *cache)
{
struct cachefiles_object *fsdef;
struct path path;
struct kstatfs stats;
struct dentry *graveyard, *cachedir, *root;
const struct cred *saved_cred;
int ret;
_enter("");
/* we want to work under the module's security ID */
ret = cachefiles_get_security_ID(cache);
if (ret < 0)
return ret;
cachefiles_begin_secure(cache, &saved_cred);
/* allocate the root index object */
ret = -ENOMEM;
fsdef = kmem_cache_alloc(cachefiles_object_jar, GFP_KERNEL);
if (!fsdef)
goto error_root_object;
ASSERTCMP(fsdef->backer, ==, NULL);
atomic_set(&fsdef->usage, 1);
fsdef->type = FSCACHE_COOKIE_TYPE_INDEX;
/* look up the directory at the root of the cache */
ret = kern_path(cache->rootdirname, LOOKUP_DIRECTORY, &path);
if (ret < 0)
goto error_open_root;
cache->mnt = path.mnt;
root = path.dentry;
ret = -EINVAL;
if (is_idmapped_mnt(path.mnt)) {
pr_warn("File cache on idmapped mounts not supported");
goto error_unsupported;
}
/* check parameters */
ret = -EOPNOTSUPP;
if (d_is_negative(root) ||
!d_backing_inode(root)->i_op->lookup ||
!d_backing_inode(root)->i_op->mkdir ||
!(d_backing_inode(root)->i_opflags & IOP_XATTR) ||
!root->d_sb->s_op->statfs ||
!root->d_sb->s_op->sync_fs)
goto error_unsupported;
ret = -EROFS;
if (sb_rdonly(root->d_sb))
goto error_unsupported;
/* determine the security of the on-disk cache as this governs
* security ID of files we create */
ret = cachefiles_determine_cache_security(cache, root, &saved_cred);
if (ret < 0)
goto error_unsupported;
/* get the cache size and blocksize */
ret = vfs_statfs(&path, &stats);
if (ret < 0)
goto error_unsupported;
ret = -ERANGE;
if (stats.f_bsize <= 0)
goto error_unsupported;
ret = -EOPNOTSUPP;
if (stats.f_bsize > PAGE_SIZE)
goto error_unsupported;
cache->bsize = stats.f_bsize;
cache->bshift = 0;
if (stats.f_bsize < PAGE_SIZE)
cache->bshift = PAGE_SHIFT - ilog2(stats.f_bsize);
_debug("blksize %u (shift %u)",
cache->bsize, cache->bshift);
_debug("size %llu, avail %llu",
(unsigned long long) stats.f_blocks,
(unsigned long long) stats.f_bavail);
/* set up caching limits */
do_div(stats.f_files, 100);
cache->fstop = stats.f_files * cache->fstop_percent;
cache->fcull = stats.f_files * cache->fcull_percent;
cache->frun = stats.f_files * cache->frun_percent;
_debug("limits {%llu,%llu,%llu} files",
(unsigned long long) cache->frun,
(unsigned long long) cache->fcull,
(unsigned long long) cache->fstop);
stats.f_blocks >>= cache->bshift;
do_div(stats.f_blocks, 100);
cache->bstop = stats.f_blocks * cache->bstop_percent;
cache->bcull = stats.f_blocks * cache->bcull_percent;
cache->brun = stats.f_blocks * cache->brun_percent;
_debug("limits {%llu,%llu,%llu} blocks",
(unsigned long long) cache->brun,
(unsigned long long) cache->bcull,
(unsigned long long) cache->bstop);
/* get the cache directory and check its type */
cachedir = cachefiles_get_directory(cache, root, "cache");
if (IS_ERR(cachedir)) {
ret = PTR_ERR(cachedir);
goto error_unsupported;
}
fsdef->dentry = cachedir;
fsdef->fscache.cookie = NULL;
ret = cachefiles_check_object_type(fsdef);
if (ret < 0)
goto error_unsupported;
/* get the graveyard directory */
graveyard = cachefiles_get_directory(cache, root, "graveyard");
if (IS_ERR(graveyard)) {
ret = PTR_ERR(graveyard);
goto error_unsupported;
}
cache->graveyard = graveyard;
/* publish the cache */
fscache_init_cache(&cache->cache,
&cachefiles_cache_ops,
"%s",
fsdef->dentry->d_sb->s_id);
fscache_object_init(&fsdef->fscache, &fscache_fsdef_index,
&cache->cache);
ret = fscache_add_cache(&cache->cache, &fsdef->fscache, cache->tag);
if (ret < 0)
goto error_add_cache;
/* done */
set_bit(CACHEFILES_READY, &cache->flags);
dput(root);
pr_info("File cache on %s registered\n", cache->cache.identifier);
/* check how much space the cache has */
cachefiles_has_space(cache, 0, 0);
cachefiles_end_secure(cache, saved_cred);
return 0;
error_add_cache:
dput(cache->graveyard);
cache->graveyard = NULL;
error_unsupported:
mntput(cache->mnt);
cache->mnt = NULL;
dput(fsdef->dentry);
fsdef->dentry = NULL;
dput(root);
error_open_root:
kmem_cache_free(cachefiles_object_jar, fsdef);
error_root_object:
cachefiles_end_secure(cache, saved_cred);
pr_err("Failed to register: %d\n", ret);
return ret;
}
/*
* unbind a cache on fd release
*/
void cachefiles_daemon_unbind(struct cachefiles_cache *cache)
{
_enter("");
if (test_bit(CACHEFILES_READY, &cache->flags)) {
pr_info("File cache on %s unregistering\n",
cache->cache.identifier);
fscache_withdraw_cache(&cache->cache);
}
dput(cache->graveyard);
mntput(cache->mnt);
kfree(cache->rootdirname);
kfree(cache->secctx);
kfree(cache->tag);
_leave("");
}

378
fs/cachefiles/cache.c Normal file
View File

@ -0,0 +1,378 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Manage high-level VFS aspects of a cache.
*
* Copyright (C) 2007, 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/slab.h>
#include <linux/statfs.h>
#include <linux/namei.h>
#include "internal.h"
/*
* Bring a cache online.
*/
int cachefiles_add_cache(struct cachefiles_cache *cache)
{
struct fscache_cache *cache_cookie;
struct path path;
struct kstatfs stats;
struct dentry *graveyard, *cachedir, *root;
const struct cred *saved_cred;
int ret;
_enter("");
cache_cookie = fscache_acquire_cache(cache->tag);
if (IS_ERR(cache_cookie))
return PTR_ERR(cache_cookie);
/* we want to work under the module's security ID */
ret = cachefiles_get_security_ID(cache);
if (ret < 0)
goto error_getsec;
cachefiles_begin_secure(cache, &saved_cred);
/* look up the directory at the root of the cache */
ret = kern_path(cache->rootdirname, LOOKUP_DIRECTORY, &path);
if (ret < 0)
goto error_open_root;
cache->mnt = path.mnt;
root = path.dentry;
ret = -EINVAL;
if (is_idmapped_mnt(path.mnt)) {
pr_warn("File cache on idmapped mounts not supported");
goto error_unsupported;
}
/* check parameters */
ret = -EOPNOTSUPP;
if (d_is_negative(root) ||
!d_backing_inode(root)->i_op->lookup ||
!d_backing_inode(root)->i_op->mkdir ||
!(d_backing_inode(root)->i_opflags & IOP_XATTR) ||
!root->d_sb->s_op->statfs ||
!root->d_sb->s_op->sync_fs ||
root->d_sb->s_blocksize > PAGE_SIZE)
goto error_unsupported;
ret = -EROFS;
if (sb_rdonly(root->d_sb))
goto error_unsupported;
/* determine the security of the on-disk cache as this governs
* security ID of files we create */
ret = cachefiles_determine_cache_security(cache, root, &saved_cred);
if (ret < 0)
goto error_unsupported;
/* get the cache size and blocksize */
ret = vfs_statfs(&path, &stats);
if (ret < 0)
goto error_unsupported;
ret = -ERANGE;
if (stats.f_bsize <= 0)
goto error_unsupported;
ret = -EOPNOTSUPP;
if (stats.f_bsize > PAGE_SIZE)
goto error_unsupported;
cache->bsize = stats.f_bsize;
cache->bshift = 0;
if (stats.f_bsize < PAGE_SIZE)
cache->bshift = PAGE_SHIFT - ilog2(stats.f_bsize);
_debug("blksize %u (shift %u)",
cache->bsize, cache->bshift);
_debug("size %llu, avail %llu",
(unsigned long long) stats.f_blocks,
(unsigned long long) stats.f_bavail);
/* set up caching limits */
do_div(stats.f_files, 100);
cache->fstop = stats.f_files * cache->fstop_percent;
cache->fcull = stats.f_files * cache->fcull_percent;
cache->frun = stats.f_files * cache->frun_percent;
_debug("limits {%llu,%llu,%llu} files",
(unsigned long long) cache->frun,
(unsigned long long) cache->fcull,
(unsigned long long) cache->fstop);
stats.f_blocks >>= cache->bshift;
do_div(stats.f_blocks, 100);
cache->bstop = stats.f_blocks * cache->bstop_percent;
cache->bcull = stats.f_blocks * cache->bcull_percent;
cache->brun = stats.f_blocks * cache->brun_percent;
_debug("limits {%llu,%llu,%llu} blocks",
(unsigned long long) cache->brun,
(unsigned long long) cache->bcull,
(unsigned long long) cache->bstop);
/* get the cache directory and check its type */
cachedir = cachefiles_get_directory(cache, root, "cache", NULL);
if (IS_ERR(cachedir)) {
ret = PTR_ERR(cachedir);
goto error_unsupported;
}
cache->store = cachedir;
/* get the graveyard directory */
graveyard = cachefiles_get_directory(cache, root, "graveyard", NULL);
if (IS_ERR(graveyard)) {
ret = PTR_ERR(graveyard);
goto error_unsupported;
}
cache->graveyard = graveyard;
cache->cache = cache_cookie;
ret = fscache_add_cache(cache_cookie, &cachefiles_cache_ops, cache);
if (ret < 0)
goto error_add_cache;
/* done */
set_bit(CACHEFILES_READY, &cache->flags);
dput(root);
pr_info("File cache on %s registered\n", cache_cookie->name);
/* check how much space the cache has */
cachefiles_has_space(cache, 0, 0, cachefiles_has_space_check);
cachefiles_end_secure(cache, saved_cred);
_leave(" = 0 [%px]", cache->cache);
return 0;
error_add_cache:
cachefiles_put_directory(cache->graveyard);
cache->graveyard = NULL;
error_unsupported:
cachefiles_put_directory(cache->store);
cache->store = NULL;
mntput(cache->mnt);
cache->mnt = NULL;
dput(root);
error_open_root:
cachefiles_end_secure(cache, saved_cred);
error_getsec:
fscache_relinquish_cache(cache_cookie);
cache->cache = NULL;
pr_err("Failed to register: %d\n", ret);
return ret;
}
/*
* See if we have space for a number of pages and/or a number of files in the
* cache
*/
int cachefiles_has_space(struct cachefiles_cache *cache,
unsigned fnr, unsigned bnr,
enum cachefiles_has_space_for reason)
{
struct kstatfs stats;
u64 b_avail, b_writing;
int ret;
struct path path = {
.mnt = cache->mnt,
.dentry = cache->mnt->mnt_root,
};
//_enter("{%llu,%llu,%llu,%llu,%llu,%llu},%u,%u",
// (unsigned long long) cache->frun,
// (unsigned long long) cache->fcull,
// (unsigned long long) cache->fstop,
// (unsigned long long) cache->brun,
// (unsigned long long) cache->bcull,
// (unsigned long long) cache->bstop,
// fnr, bnr);
/* find out how many pages of blockdev are available */
memset(&stats, 0, sizeof(stats));
ret = vfs_statfs(&path, &stats);
if (ret < 0) {
trace_cachefiles_vfs_error(NULL, d_inode(path.dentry), ret,
cachefiles_trace_statfs_error);
if (ret == -EIO)
cachefiles_io_error(cache, "statfs failed");
_leave(" = %d", ret);
return ret;
}
b_avail = stats.f_bavail >> cache->bshift;
b_writing = atomic_long_read(&cache->b_writing);
if (b_avail > b_writing)
b_avail -= b_writing;
else
b_avail = 0;
//_debug("avail %llu,%llu",
// (unsigned long long)stats.f_ffree,
// (unsigned long long)b_avail);
/* see if there is sufficient space */
if (stats.f_ffree > fnr)
stats.f_ffree -= fnr;
else
stats.f_ffree = 0;
if (b_avail > bnr)
b_avail -= bnr;
else
b_avail = 0;
ret = -ENOBUFS;
if (stats.f_ffree < cache->fstop ||
b_avail < cache->bstop)
goto stop_and_begin_cull;
ret = 0;
if (stats.f_ffree < cache->fcull ||
b_avail < cache->bcull)
goto begin_cull;
if (test_bit(CACHEFILES_CULLING, &cache->flags) &&
stats.f_ffree >= cache->frun &&
b_avail >= cache->brun &&
test_and_clear_bit(CACHEFILES_CULLING, &cache->flags)
) {
_debug("cease culling");
cachefiles_state_changed(cache);
}
//_leave(" = 0");
return 0;
stop_and_begin_cull:
switch (reason) {
case cachefiles_has_space_for_write:
fscache_count_no_write_space();
break;
case cachefiles_has_space_for_create:
fscache_count_no_create_space();
break;
default:
break;
}
begin_cull:
if (!test_and_set_bit(CACHEFILES_CULLING, &cache->flags)) {
_debug("### CULL CACHE ###");
cachefiles_state_changed(cache);
}
_leave(" = %d", ret);
return ret;
}
/*
* Mark all the objects as being out of service and queue them all for cleanup.
*/
static void cachefiles_withdraw_objects(struct cachefiles_cache *cache)
{
struct cachefiles_object *object;
unsigned int count = 0;
_enter("");
spin_lock(&cache->object_list_lock);
while (!list_empty(&cache->object_list)) {
object = list_first_entry(&cache->object_list,
struct cachefiles_object, cache_link);
cachefiles_see_object(object, cachefiles_obj_see_withdrawal);
list_del_init(&object->cache_link);
fscache_withdraw_cookie(object->cookie);
count++;
if ((count & 63) == 0) {
spin_unlock(&cache->object_list_lock);
cond_resched();
spin_lock(&cache->object_list_lock);
}
}
spin_unlock(&cache->object_list_lock);
_leave(" [%u objs]", count);
}
/*
* Withdraw volumes.
*/
static void cachefiles_withdraw_volumes(struct cachefiles_cache *cache)
{
_enter("");
for (;;) {
struct cachefiles_volume *volume = NULL;
spin_lock(&cache->object_list_lock);
if (!list_empty(&cache->volumes)) {
volume = list_first_entry(&cache->volumes,
struct cachefiles_volume, cache_link);
list_del_init(&volume->cache_link);
}
spin_unlock(&cache->object_list_lock);
if (!volume)
break;
cachefiles_withdraw_volume(volume);
}
_leave("");
}
/*
* Sync a cache to backing disk.
*/
static void cachefiles_sync_cache(struct cachefiles_cache *cache)
{
const struct cred *saved_cred;
int ret;
_enter("%s", cache->cache->name);
/* make sure all pages pinned by operations on behalf of the netfs are
* written to disc */
cachefiles_begin_secure(cache, &saved_cred);
down_read(&cache->mnt->mnt_sb->s_umount);
ret = sync_filesystem(cache->mnt->mnt_sb);
up_read(&cache->mnt->mnt_sb->s_umount);
cachefiles_end_secure(cache, saved_cred);
if (ret == -EIO)
cachefiles_io_error(cache,
"Attempt to sync backing fs superblock returned error %d",
ret);
}
/*
* Withdraw cache objects.
*/
void cachefiles_withdraw_cache(struct cachefiles_cache *cache)
{
struct fscache_cache *fscache = cache->cache;
pr_info("File cache on %s unregistering\n", fscache->name);
fscache_withdraw_cache(fscache);
/* we now have to destroy all the active objects pertaining to this
* cache - which we do by passing them off to thread pool to be
* disposed of */
cachefiles_withdraw_objects(cache);
fscache_wait_for_objects(fscache);
cachefiles_withdraw_volumes(cache);
cachefiles_sync_cache(cache);
cache->cache = NULL;
fscache_relinquish_cache(fscache);
}

176
fs/cachefiles/daemon.c
View File

@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Daemon interface
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2007, 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
@ -41,6 +41,8 @@ static int cachefiles_daemon_dir(struct cachefiles_cache *, char *);
static int cachefiles_daemon_inuse(struct cachefiles_cache *, char *);
static int cachefiles_daemon_secctx(struct cachefiles_cache *, char *);
static int cachefiles_daemon_tag(struct cachefiles_cache *, char *);
static int cachefiles_daemon_bind(struct cachefiles_cache *, char *);
static void cachefiles_daemon_unbind(struct cachefiles_cache *);
static unsigned long cachefiles_open;
@ -78,7 +80,7 @@ static const struct cachefiles_daemon_cmd cachefiles_daemon_cmds[] = {
/*
* do various checks
* Prepare a cache for caching.
*/
static int cachefiles_daemon_open(struct inode *inode, struct file *file)
{
@ -102,9 +104,10 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
}
mutex_init(&cache->daemon_mutex);
cache->active_nodes = RB_ROOT;
rwlock_init(&cache->active_lock);
init_waitqueue_head(&cache->daemon_pollwq);
INIT_LIST_HEAD(&cache->volumes);
INIT_LIST_HEAD(&cache->object_list);
spin_lock_init(&cache->object_list_lock);
/* set default caching limits
* - limit at 1% free space and/or free files
@ -124,7 +127,7 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
}
/*
* release a cache
* Release a cache.
*/
static int cachefiles_daemon_release(struct inode *inode, struct file *file)
{
@ -138,8 +141,6 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
cachefiles_daemon_unbind(cache);
ASSERT(!cache->active_nodes.rb_node);
/* clean up the control file interface */
cache->cachefilesd = NULL;
file->private_data = NULL;
@ -152,7 +153,7 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
}
/*
* read the cache state
* Read the cache state.
*/
static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
size_t buflen, loff_t *pos)
@ -169,7 +170,7 @@ static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
return 0;
/* check how much space the cache has */
cachefiles_has_space(cache, 0, 0);
cachefiles_has_space(cache, 0, 0, cachefiles_has_space_check);
/* summarise */
f_released = atomic_xchg(&cache->f_released, 0);
@ -206,7 +207,7 @@ static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
}
/*
* command the cache
* Take a command from cachefilesd, parse it and act on it.
*/
static ssize_t cachefiles_daemon_write(struct file *file,
const char __user *_data,
@ -225,7 +226,7 @@ static ssize_t cachefiles_daemon_write(struct file *file,
if (test_bit(CACHEFILES_DEAD, &cache->flags))
return -EIO;
if (datalen < 0 || datalen > PAGE_SIZE - 1)
if (datalen > PAGE_SIZE - 1)
return -EOPNOTSUPP;
/* drag the command string into the kernel so we can parse it */
@ -284,7 +285,7 @@ static ssize_t cachefiles_daemon_write(struct file *file,
}
/*
* poll for culling state
* Poll for culling state
* - use EPOLLOUT to indicate culling state
*/
static __poll_t cachefiles_daemon_poll(struct file *file,
@ -306,7 +307,7 @@ static __poll_t cachefiles_daemon_poll(struct file *file,
}
/*
* give a range error for cache space constraints
* Give a range error for cache space constraints
* - can be tail-called
*/
static int cachefiles_daemon_range_error(struct cachefiles_cache *cache,
@ -318,7 +319,7 @@ static int cachefiles_daemon_range_error(struct cachefiles_cache *cache,
}
/*
* set the percentage of files at which to stop culling
* Set the percentage of files at which to stop culling
* - command: "frun <N>%"
*/
static int cachefiles_daemon_frun(struct cachefiles_cache *cache, char *args)
@ -342,7 +343,7 @@ static int cachefiles_daemon_frun(struct cachefiles_cache *cache, char *args)
}
/*
* set the percentage of files at which to start culling
* Set the percentage of files at which to start culling
* - command: "fcull <N>%"
*/
static int cachefiles_daemon_fcull(struct cachefiles_cache *cache, char *args)
@ -366,7 +367,7 @@ static int cachefiles_daemon_fcull(struct cachefiles_cache *cache, char *args)
}
/*
* set the percentage of files at which to stop allocating
* Set the percentage of files at which to stop allocating
* - command: "fstop <N>%"
*/
static int cachefiles_daemon_fstop(struct cachefiles_cache *cache, char *args)
@ -382,7 +383,7 @@ static int cachefiles_daemon_fstop(struct cachefiles_cache *cache, char *args)
if (args[0] != '%' || args[1] != '\0')
return -EINVAL;
if (fstop < 0 || fstop >= cache->fcull_percent)
if (fstop >= cache->fcull_percent)
return cachefiles_daemon_range_error(cache, args);
cache->fstop_percent = fstop;
@ -390,7 +391,7 @@ static int cachefiles_daemon_fstop(struct cachefiles_cache *cache, char *args)
}
/*
* set the percentage of blocks at which to stop culling
* Set the percentage of blocks at which to stop culling
* - command: "brun <N>%"
*/
static int cachefiles_daemon_brun(struct cachefiles_cache *cache, char *args)
@ -414,7 +415,7 @@ static int cachefiles_daemon_brun(struct cachefiles_cache *cache, char *args)
}
/*
* set the percentage of blocks at which to start culling
* Set the percentage of blocks at which to start culling
* - command: "bcull <N>%"
*/
static int cachefiles_daemon_bcull(struct cachefiles_cache *cache, char *args)
@ -438,7 +439,7 @@ static int cachefiles_daemon_bcull(struct cachefiles_cache *cache, char *args)
}
/*
* set the percentage of blocks at which to stop allocating
* Set the percentage of blocks at which to stop allocating
* - command: "bstop <N>%"
*/
static int cachefiles_daemon_bstop(struct cachefiles_cache *cache, char *args)
@ -454,7 +455,7 @@ static int cachefiles_daemon_bstop(struct cachefiles_cache *cache, char *args)
if (args[0] != '%' || args[1] != '\0')
return -EINVAL;
if (bstop < 0 || bstop >= cache->bcull_percent)
if (bstop >= cache->bcull_percent)
return cachefiles_daemon_range_error(cache, args);
cache->bstop_percent = bstop;
@ -462,7 +463,7 @@ static int cachefiles_daemon_bstop(struct cachefiles_cache *cache, char *args)
}
/*
* set the cache directory
* Set the cache directory
* - command: "dir <name>"
*/
static int cachefiles_daemon_dir(struct cachefiles_cache *cache, char *args)
@ -490,7 +491,7 @@ static int cachefiles_daemon_dir(struct cachefiles_cache *cache, char *args)
}
/*
* set the cache security context
* Set the cache security context
* - command: "secctx <ctx>"
*/
static int cachefiles_daemon_secctx(struct cachefiles_cache *cache, char *args)
@ -518,7 +519,7 @@ static int cachefiles_daemon_secctx(struct cachefiles_cache *cache, char *args)
}
/*
* set the cache tag
* Set the cache tag
* - command: "tag <name>"
*/
static int cachefiles_daemon_tag(struct cachefiles_cache *cache, char *args)
@ -544,7 +545,7 @@ static int cachefiles_daemon_tag(struct cachefiles_cache *cache, char *args)
}
/*
* request a node in the cache be culled from the current working directory
* Request a node in the cache be culled from the current working directory
* - command: "cull <name>"
*/
static int cachefiles_daemon_cull(struct cachefiles_cache *cache, char *args)
@ -568,7 +569,6 @@ static int cachefiles_daemon_cull(struct cachefiles_cache *cache, char *args)
return -EIO;
}
/* extract the directory dentry from the cwd */
get_fs_pwd(current->fs, &path);
if (!d_can_lookup(path.dentry))
@ -593,7 +593,7 @@ static int cachefiles_daemon_cull(struct cachefiles_cache *cache, char *args)
}
/*
* set debugging mode
* Set debugging mode
* - command: "debug <mask>"
*/
static int cachefiles_daemon_debug(struct cachefiles_cache *cache, char *args)
@ -616,7 +616,7 @@ static int cachefiles_daemon_debug(struct cachefiles_cache *cache, char *args)
}
/*
* find out whether an object in the current working directory is in use or not
* Find out whether an object in the current working directory is in use or not
* - command: "inuse <name>"
*/
static int cachefiles_daemon_inuse(struct cachefiles_cache *cache, char *args)
@ -640,7 +640,6 @@ static int cachefiles_daemon_inuse(struct cachefiles_cache *cache, char *args)
return -EIO;
}
/* extract the directory dentry from the cwd */
get_fs_pwd(current->fs, &path);
if (!d_can_lookup(path.dentry))
@ -665,84 +664,65 @@ static int cachefiles_daemon_inuse(struct cachefiles_cache *cache, char *args)
}
/*
* see if we have space for a number of pages and/or a number of files in the
* cache
* Bind a directory as a cache
*/
int cachefiles_has_space(struct cachefiles_cache *cache,
unsigned fnr, unsigned bnr)
static int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
{
struct kstatfs stats;
struct path path = {
.mnt = cache->mnt,
.dentry = cache->mnt->mnt_root,
};
int ret;
_enter("{%u,%u,%u,%u,%u,%u},%s",
cache->frun_percent,
cache->fcull_percent,
cache->fstop_percent,
cache->brun_percent,
cache->bcull_percent,
cache->bstop_percent,
args);
//_enter("{%llu,%llu,%llu,%llu,%llu,%llu},%u,%u",
// (unsigned long long) cache->frun,
// (unsigned long long) cache->fcull,
// (unsigned long long) cache->fstop,
// (unsigned long long) cache->brun,
// (unsigned long long) cache->bcull,
// (unsigned long long) cache->bstop,
// fnr, bnr);
if (cache->fstop_percent >= cache->fcull_percent ||
cache->fcull_percent >= cache->frun_percent ||
cache->frun_percent >= 100)
return -ERANGE;
/* find out how many pages of blockdev are available */
memset(&stats, 0, sizeof(stats));
if (cache->bstop_percent >= cache->bcull_percent ||
cache->bcull_percent >= cache->brun_percent ||
cache->brun_percent >= 100)
return -ERANGE;
ret = vfs_statfs(&path, &stats);
if (ret < 0) {
if (ret == -EIO)
cachefiles_io_error(cache, "statfs failed");
_leave(" = %d", ret);
return ret;
if (*args) {
pr_err("'bind' command doesn't take an argument\n");
return -EINVAL;
}
stats.f_bavail >>= cache->bshift;
//_debug("avail %llu,%llu",
// (unsigned long long) stats.f_ffree,
// (unsigned long long) stats.f_bavail);
/* see if there is sufficient space */
if (stats.f_ffree > fnr)
stats.f_ffree -= fnr;
else
stats.f_ffree = 0;
if (stats.f_bavail > bnr)
stats.f_bavail -= bnr;
else
stats.f_bavail = 0;
ret = -ENOBUFS;
if (stats.f_ffree < cache->fstop ||
stats.f_bavail < cache->bstop)
goto begin_cull;
ret = 0;
if (stats.f_ffree < cache->fcull ||
stats.f_bavail < cache->bcull)
goto begin_cull;
if (test_bit(CACHEFILES_CULLING, &cache->flags) &&
stats.f_ffree >= cache->frun &&
stats.f_bavail >= cache->brun &&
test_and_clear_bit(CACHEFILES_CULLING, &cache->flags)
) {
_debug("cease culling");
cachefiles_state_changed(cache);
if (!cache->rootdirname) {
pr_err("No cache directory specified\n");
return -EINVAL;
}
//_leave(" = 0");
return 0;
begin_cull:
if (!test_and_set_bit(CACHEFILES_CULLING, &cache->flags)) {
_debug("### CULL CACHE ###");
cachefiles_state_changed(cache);
/* Don't permit already bound caches to be re-bound */
if (test_bit(CACHEFILES_READY, &cache->flags)) {
pr_err("Cache already bound\n");
return -EBUSY;
}
_leave(" = %d", ret);
return ret;
return cachefiles_add_cache(cache);
}
/*
* Unbind a cache.
*/
static void cachefiles_daemon_unbind(struct cachefiles_cache *cache)
{
_enter("");
if (test_bit(CACHEFILES_READY, &cache->flags))
cachefiles_withdraw_cache(cache);
cachefiles_put_directory(cache->graveyard);
cachefiles_put_directory(cache->store);
mntput(cache->mnt);
kfree(cache->rootdirname);
kfree(cache->secctx);
kfree(cache->tag);
_leave("");
}

46
fs/cachefiles/error_inject.c Normal file
View File

@ -0,0 +1,46 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Error injection handling.
*
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/sysctl.h>
#include "internal.h"
unsigned int cachefiles_error_injection_state;
static struct ctl_table_header *cachefiles_sysctl;
static struct ctl_table cachefiles_sysctls[] = {
{
.procname = "error_injection",
.data = &cachefiles_error_injection_state,
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_douintvec,
},
{}
};
static struct ctl_table cachefiles_sysctls_root[] = {
{
.procname = "cachefiles",
.mode = 0555,
.child = cachefiles_sysctls,
},
{}
};
int __init cachefiles_register_error_injection(void)
{
cachefiles_sysctl = register_sysctl_table(cachefiles_sysctls_root);
if (!cachefiles_sysctl)
return -ENOMEM;
return 0;
}
void cachefiles_unregister_error_injection(void)
{
unregister_sysctl_table(cachefiles_sysctl);
}

793
fs/cachefiles/interface.c
View File

@ -1,467 +1,133 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache interface to CacheFiles
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/xattr.h>
#include <linux/file.h>
#include <linux/falloc.h>
#include <trace/events/fscache.h>
#include "internal.h"
struct cachefiles_lookup_data {
struct cachefiles_xattr *auxdata; /* auxiliary data */
char *key; /* key path */
};
static int cachefiles_attr_changed(struct fscache_object *_object);
static atomic_t cachefiles_object_debug_id;
/*
* allocate an object record for a cookie lookup and prepare the lookup data
*/
static struct fscache_object *cachefiles_alloc_object(
struct fscache_cache *_cache,
struct fscache_cookie *cookie)
{
struct cachefiles_lookup_data *lookup_data;
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct cachefiles_xattr *auxdata;
unsigned keylen, auxlen;
void *buffer, *p;
char *key;
cache = container_of(_cache, struct cachefiles_cache, cache);
_enter("{%s},%x,", cache->cache.identifier, cookie->debug_id);
lookup_data = kmalloc(sizeof(*lookup_data), cachefiles_gfp);
if (!lookup_data)
goto nomem_lookup_data;
/* create a new object record and a temporary leaf image */
object = kmem_cache_alloc(cachefiles_object_jar, cachefiles_gfp);
if (!object)
goto nomem_object;
ASSERTCMP(object->backer, ==, NULL);
BUG_ON(test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
atomic_set(&object->usage, 1);
fscache_object_init(&object->fscache, cookie, &cache->cache);
object->type = cookie->def->type;
/* get hold of the raw key
* - stick the length on the front and leave space on the back for the
* encoder
*/
buffer = kmalloc((2 + 512) + 3, cachefiles_gfp);
if (!buffer)
goto nomem_buffer;
keylen = cookie->key_len;
if (keylen <= sizeof(cookie->inline_key))
p = cookie->inline_key;
else
p = cookie->key;
memcpy(buffer + 2, p, keylen);
*(uint16_t *)buffer = keylen;
((char *)buffer)[keylen + 2] = 0;
((char *)buffer)[keylen + 3] = 0;
((char *)buffer)[keylen + 4] = 0;
/* turn the raw key into something that can work with as a filename */
key = cachefiles_cook_key(buffer, keylen + 2, object->type);
if (!key)
goto nomem_key;
/* get hold of the auxiliary data and prepend the object type */
auxdata = buffer;
auxlen = cookie->aux_len;
if (auxlen) {
if (auxlen <= sizeof(cookie->inline_aux))
p = cookie->inline_aux;
else
p = cookie->aux;
memcpy(auxdata->data, p, auxlen);
}
auxdata->len = auxlen + 1;
auxdata->type = cookie->type;
lookup_data->auxdata = auxdata;
lookup_data->key = key;
object->lookup_data = lookup_data;
_leave(" = %x [%p]", object->fscache.debug_id, lookup_data);
return &object->fscache;
nomem_key:
kfree(buffer);
nomem_buffer:
BUG_ON(test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
kmem_cache_free(cachefiles_object_jar, object);
fscache_object_destroyed(&cache->cache);
nomem_object:
kfree(lookup_data);
nomem_lookup_data:
_leave(" = -ENOMEM");
return ERR_PTR(-ENOMEM);
}
/*
* attempt to look up the nominated node in this cache
* - return -ETIMEDOUT to be scheduled again
*/
static int cachefiles_lookup_object(struct fscache_object *_object)
{
struct cachefiles_lookup_data *lookup_data;
struct cachefiles_object *parent, *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
int ret;
_enter("{OBJ%x}", _object->debug_id);
cache = container_of(_object->cache, struct cachefiles_cache, cache);
parent = container_of(_object->parent,
struct cachefiles_object, fscache);
object = container_of(_object, struct cachefiles_object, fscache);
lookup_data = object->lookup_data;
ASSERTCMP(lookup_data, !=, NULL);
/* look up the key, creating any missing bits */
cachefiles_begin_secure(cache, &saved_cred);
ret = cachefiles_walk_to_object(parent, object,
lookup_data->key,
lookup_data->auxdata);
cachefiles_end_secure(cache, saved_cred);
/* polish off by setting the attributes of non-index files */
if (ret == 0 &&
object->fscache.cookie->def->type != FSCACHE_COOKIE_TYPE_INDEX)
cachefiles_attr_changed(&object->fscache);
if (ret < 0 && ret != -ETIMEDOUT) {
if (ret != -ENOBUFS)
pr_warn("Lookup failed error %d\n", ret);
fscache_object_lookup_error(&object->fscache);
}
_leave(" [%d]", ret);
return ret;
}
/*
* indication of lookup completion
*/
static void cachefiles_lookup_complete(struct fscache_object *_object)
{
struct cachefiles_object *object;
object = container_of(_object, struct cachefiles_object, fscache);
_enter("{OBJ%x,%p}", object->fscache.debug_id, object->lookup_data);
if (object->lookup_data) {
kfree(object->lookup_data->key);
kfree(object->lookup_data->auxdata);
kfree(object->lookup_data);
object->lookup_data = NULL;
}
}
/*
* increment the usage count on an inode object (may fail if unmounting)
* Allocate a cache object record.
*/
static
struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
enum fscache_obj_ref_trace why)
struct cachefiles_object *cachefiles_alloc_object(struct fscache_cookie *cookie)
{
struct cachefiles_object *object =
container_of(_object, struct cachefiles_object, fscache);
int u;
struct fscache_volume *vcookie = cookie->volume;
struct cachefiles_volume *volume = vcookie->cache_priv;
struct cachefiles_object *object;
_enter("{OBJ%x,%d}", _object->debug_id, atomic_read(&object->usage));
_enter("{%s},%x,", vcookie->key, cookie->debug_id);
#ifdef CACHEFILES_DEBUG_SLAB
ASSERT((atomic_read(&object->usage) & 0xffff0000) != 0x6b6b0000);
#endif
object = kmem_cache_zalloc(cachefiles_object_jar, GFP_KERNEL);
if (!object)
return NULL;
u = atomic_inc_return(&object->usage);
trace_cachefiles_ref(object, _object->cookie,
(enum cachefiles_obj_ref_trace)why, u);
return &object->fscache;
refcount_set(&object->ref, 1);
spin_lock_init(&object->lock);
INIT_LIST_HEAD(&object->cache_link);
object->volume = volume;
object->debug_id = atomic_inc_return(&cachefiles_object_debug_id);
object->cookie = fscache_get_cookie(cookie, fscache_cookie_get_attach_object);
fscache_count_object(vcookie->cache);
trace_cachefiles_ref(object->debug_id, cookie->debug_id, 1,
cachefiles_obj_new);
return object;
}
/*
* update the auxiliary data for an object object on disk
* Note that an object has been seen.
*/
static void cachefiles_update_object(struct fscache_object *_object)
void cachefiles_see_object(struct cachefiles_object *object,
enum cachefiles_obj_ref_trace why)
{
struct cachefiles_object *object;
struct cachefiles_xattr *auxdata;
struct cachefiles_cache *cache;
struct fscache_cookie *cookie;
const struct cred *saved_cred;
const void *aux;
unsigned auxlen;
_enter("{OBJ%x}", _object->debug_id);
object = container_of(_object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache, struct cachefiles_cache,
cache);
if (!fscache_use_cookie(_object)) {
_leave(" [relinq]");
return;
}
cookie = object->fscache.cookie;
auxlen = cookie->aux_len;
if (!auxlen) {
fscache_unuse_cookie(_object);
_leave(" [no aux]");
return;
}
auxdata = kmalloc(2 + auxlen + 3, cachefiles_gfp);
if (!auxdata) {
fscache_unuse_cookie(_object);
_leave(" [nomem]");
return;
}
aux = (auxlen <= sizeof(cookie->inline_aux)) ?
cookie->inline_aux : cookie->aux;
memcpy(auxdata->data, aux, auxlen);
fscache_unuse_cookie(_object);
auxdata->len = auxlen + 1;
auxdata->type = cookie->type;
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_update_object_xattr(object, auxdata);
cachefiles_end_secure(cache, saved_cred);
kfree(auxdata);
_leave("");
trace_cachefiles_ref(object->debug_id, object->cookie->debug_id,
refcount_read(&object->ref), why);
}
/*
* discard the resources pinned by an object and effect retirement if
* requested
* Increment the usage count on an object;
*/
static void cachefiles_drop_object(struct fscache_object *_object)
struct cachefiles_object *cachefiles_grab_object(struct cachefiles_object *object,
enum cachefiles_obj_ref_trace why)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
struct inode *inode;
blkcnt_t i_blocks = 0;
int r;
ASSERT(_object);
object = container_of(_object, struct cachefiles_object, fscache);
_enter("{OBJ%x,%d}",
object->fscache.debug_id, atomic_read(&object->usage));
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
#ifdef CACHEFILES_DEBUG_SLAB
ASSERT((atomic_read(&object->usage) & 0xffff0000) != 0x6b6b0000);
#endif
/* We need to tidy the object up if we did in fact manage to open it.
* It's possible for us to get here before the object is fully
* initialised if the parent goes away or the object gets retired
* before we set it up.
*/
if (object->dentry) {
/* delete retired objects */
if (test_bit(FSCACHE_OBJECT_RETIRED, &object->fscache.flags) &&
_object != cache->cache.fsdef
) {
_debug("- retire object OBJ%x", object->fscache.debug_id);
inode = d_backing_inode(object->dentry);
if (inode)
i_blocks = inode->i_blocks;
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_delete_object(cache, object);
cachefiles_end_secure(cache, saved_cred);
}
/* close the filesystem stuff attached to the object */
if (object->backer != object->dentry)
dput(object->backer);
object->backer = NULL;
}
/* note that the object is now inactive */
if (test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags))
cachefiles_mark_object_inactive(cache, object, i_blocks);
dput(object->dentry);
object->dentry = NULL;
_leave("");
__refcount_inc(&object->ref, &r);
trace_cachefiles_ref(object->debug_id, object->cookie->debug_id, r, why);
return object;
}
/*
* dispose of a reference to an object
*/
void cachefiles_put_object(struct fscache_object *_object,
enum fscache_obj_ref_trace why)
void cachefiles_put_object(struct cachefiles_object *object,
enum cachefiles_obj_ref_trace why)
{
struct cachefiles_object *object;
unsigned int object_debug_id = object->debug_id;
unsigned int cookie_debug_id = object->cookie->debug_id;
struct fscache_cache *cache;
int u;
bool done;
int r;
ASSERT(_object);
done = __refcount_dec_and_test(&object->ref, &r);
trace_cachefiles_ref(object_debug_id, cookie_debug_id, r, why);
if (done) {
_debug("- kill object OBJ%x", object_debug_id);
object = container_of(_object, struct cachefiles_object, fscache);
ASSERTCMP(object->file, ==, NULL);
_enter("{OBJ%x,%d}",
object->fscache.debug_id, atomic_read(&object->usage));
kfree(object->d_name);
#ifdef CACHEFILES_DEBUG_SLAB
ASSERT((atomic_read(&object->usage) & 0xffff0000) != 0x6b6b0000);
#endif
ASSERTIFCMP(object->fscache.parent,
object->fscache.parent->n_children, >, 0);
u = atomic_dec_return(&object->usage);
trace_cachefiles_ref(object, _object->cookie,
(enum cachefiles_obj_ref_trace)why, u);
ASSERTCMP(u, !=, -1);
if (u == 0) {
_debug("- kill object OBJ%x", object->fscache.debug_id);
ASSERT(!test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
ASSERTCMP(object->fscache.parent, ==, NULL);
ASSERTCMP(object->backer, ==, NULL);
ASSERTCMP(object->dentry, ==, NULL);
ASSERTCMP(object->fscache.n_ops, ==, 0);
ASSERTCMP(object->fscache.n_children, ==, 0);
if (object->lookup_data) {
kfree(object->lookup_data->key);
kfree(object->lookup_data->auxdata);
kfree(object->lookup_data);
object->lookup_data = NULL;
}
cache = object->fscache.cache;
fscache_object_destroy(&object->fscache);
cache = object->volume->cache->cache;
fscache_put_cookie(object->cookie, fscache_cookie_put_object);
object->cookie = NULL;
kmem_cache_free(cachefiles_object_jar, object);
fscache_object_destroyed(cache);
fscache_uncount_object(cache);
}
_leave("");
}
/*
* sync a cache
* Adjust the size of a cache file if necessary to match the DIO size. We keep
* the EOF marker a multiple of DIO blocks so that we don't fall back to doing
* non-DIO for a partial block straddling the EOF, but we also have to be
* careful of someone expanding the file and accidentally accreting the
* padding.
*/
static void cachefiles_sync_cache(struct fscache_cache *_cache)
static int cachefiles_adjust_size(struct cachefiles_object *object)
{
struct cachefiles_cache *cache;
const struct cred *saved_cred;
int ret;
_enter("%s", _cache->tag->name);
cache = container_of(_cache, struct cachefiles_cache, cache);
/* make sure all pages pinned by operations on behalf of the netfs are
* written to disc */
cachefiles_begin_secure(cache, &saved_cred);
down_read(&cache->mnt->mnt_sb->s_umount);
ret = sync_filesystem(cache->mnt->mnt_sb);
up_read(&cache->mnt->mnt_sb->s_umount);
cachefiles_end_secure(cache, saved_cred);
if (ret == -EIO)
cachefiles_io_error(cache,
"Attempt to sync backing fs superblock"
" returned error %d",
ret);
}
/*
* check if the backing cache is updated to FS-Cache
* - called by FS-Cache when evaluates if need to invalidate the cache
*/
static int cachefiles_check_consistency(struct fscache_operation *op)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
int ret;
_enter("{OBJ%x}", op->object->debug_id);
object = container_of(op->object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
cachefiles_begin_secure(cache, &saved_cred);
ret = cachefiles_check_auxdata(object);
cachefiles_end_secure(cache, saved_cred);
_leave(" = %d", ret);
return ret;
}
/*
* notification the attributes on an object have changed
* - called with reads/writes excluded by FS-Cache
*/
static int cachefiles_attr_changed(struct fscache_object *_object)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
struct iattr newattrs;
struct file *file = object->file;
uint64_t ni_size;
loff_t oi_size;
int ret;
ni_size = _object->store_limit_l;
ni_size = object->cookie->object_size;
ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
_enter("{OBJ%x},[%llu]",
_object->debug_id, (unsigned long long) ni_size);
object->debug_id, (unsigned long long) ni_size);
object = container_of(_object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
if (ni_size == object->i_size)
return 0;
if (!object->backer)
if (!file)
return -ENOBUFS;
ASSERT(d_is_reg(object->backer));
fscache_set_store_limit(&object->fscache, ni_size);
oi_size = i_size_read(d_backing_inode(object->backer));
oi_size = i_size_read(file_inode(file));
if (oi_size == ni_size)
return 0;
cachefiles_begin_secure(cache, &saved_cred);
inode_lock(d_inode(object->backer));
inode_lock(file_inode(file));
/* if there's an extension to a partial page at the end of the backing
* file, we need to discard the partial page so that we pick up new
@ -470,21 +136,28 @@ static int cachefiles_attr_changed(struct fscache_object *_object)
_debug("discard tail %llx", oi_size);
newattrs.ia_valid = ATTR_SIZE;
newattrs.ia_size = oi_size & PAGE_MASK;
ret = notify_change(&init_user_ns, object->backer, &newattrs, NULL);
ret = cachefiles_inject_remove_error();
if (ret == 0)
ret = notify_change(&init_user_ns, file->f_path.dentry,
&newattrs, NULL);
if (ret < 0)
goto truncate_failed;
}
newattrs.ia_valid = ATTR_SIZE;
newattrs.ia_size = ni_size;
ret = notify_change(&init_user_ns, object->backer, &newattrs, NULL);
ret = cachefiles_inject_write_error();
if (ret == 0)
ret = notify_change(&init_user_ns, file->f_path.dentry,
&newattrs, NULL);
truncate_failed:
inode_unlock(d_inode(object->backer));
cachefiles_end_secure(cache, saved_cred);
inode_unlock(file_inode(file));
if (ret < 0)
trace_cachefiles_io_error(NULL, file_inode(file), ret,
cachefiles_trace_notify_change_error);
if (ret == -EIO) {
fscache_set_store_limit(&object->fscache, 0);
cachefiles_io_error_obj(object, "Size set failed");
ret = -ENOBUFS;
}
@ -494,79 +167,279 @@ static int cachefiles_attr_changed(struct fscache_object *_object)
}
/*
* Invalidate an object
* Attempt to look up the nominated node in this cache
*/
static void cachefiles_invalidate_object(struct fscache_operation *op)
static bool cachefiles_lookup_cookie(struct fscache_cookie *cookie)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct cachefiles_cache *cache = cookie->volume->cache->cache_priv;
const struct cred *saved_cred;
struct path path;
uint64_t ni_size;
int ret;
bool success;
object = container_of(op->object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
object = cachefiles_alloc_object(cookie);
if (!object)
goto fail;
ni_size = op->object->store_limit_l;
_enter("{OBJ%x}", object->debug_id);
_enter("{OBJ%x},[%llu]",
op->object->debug_id, (unsigned long long)ni_size);
if (!cachefiles_cook_key(object))
goto fail_put;
if (object->backer) {
ASSERT(d_is_reg(object->backer));
cookie->cache_priv = object;
fscache_set_store_limit(&object->fscache, ni_size);
cachefiles_begin_secure(cache, &saved_cred);
path.dentry = object->backer;
path.mnt = cache->mnt;
success = cachefiles_look_up_object(object);
if (!success)
goto fail_withdraw;
cachefiles_begin_secure(cache, &saved_cred);
ret = vfs_truncate(&path, 0);
if (ret == 0)
ret = vfs_truncate(&path, ni_size);
cachefiles_end_secure(cache, saved_cred);
cachefiles_see_object(object, cachefiles_obj_see_lookup_cookie);
if (ret != 0) {
fscache_set_store_limit(&object->fscache, 0);
if (ret == -EIO)
cachefiles_io_error_obj(object,
"Invalidate failed");
}
}
spin_lock(&cache->object_list_lock);
list_add(&object->cache_link, &cache->object_list);
spin_unlock(&cache->object_list_lock);
cachefiles_adjust_size(object);
fscache_op_complete(op, true);
_leave("");
cachefiles_end_secure(cache, saved_cred);
_leave(" = t");
return true;
fail_withdraw:
cachefiles_end_secure(cache, saved_cred);
cachefiles_see_object(object, cachefiles_obj_see_lookup_failed);
fscache_caching_failed(cookie);
_debug("failed c=%08x o=%08x", cookie->debug_id, object->debug_id);
/* The caller holds an access count on the cookie, so we need them to
* drop it before we can withdraw the object.
*/
return false;
fail_put:
cachefiles_put_object(object, cachefiles_obj_put_alloc_fail);
fail:
return false;
}
/*
* dissociate a cache from all the pages it was backing
* Shorten the backing object to discard any dirty data and free up
* any unused granules.
*/
static void cachefiles_dissociate_pages(struct fscache_cache *cache)
static bool cachefiles_shorten_object(struct cachefiles_object *object,
struct file *file, loff_t new_size)
{
_enter("");
struct cachefiles_cache *cache = object->volume->cache;
struct inode *inode = file_inode(file);
loff_t i_size, dio_size;
int ret;
dio_size = round_up(new_size, CACHEFILES_DIO_BLOCK_SIZE);
i_size = i_size_read(inode);
trace_cachefiles_trunc(object, inode, i_size, dio_size,
cachefiles_trunc_shrink);
ret = cachefiles_inject_remove_error();
if (ret == 0)
ret = vfs_truncate(&file->f_path, dio_size);
if (ret < 0) {
trace_cachefiles_io_error(object, file_inode(file), ret,
cachefiles_trace_trunc_error);
cachefiles_io_error_obj(object, "Trunc-to-size failed %d", ret);
cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
return false;
}
if (new_size < dio_size) {
trace_cachefiles_trunc(object, inode, dio_size, new_size,
cachefiles_trunc_dio_adjust);
ret = cachefiles_inject_write_error();
if (ret == 0)
ret = vfs_fallocate(file, FALLOC_FL_ZERO_RANGE,
new_size, dio_size);
if (ret < 0) {
trace_cachefiles_io_error(object, file_inode(file), ret,
cachefiles_trace_fallocate_error);
cachefiles_io_error_obj(object, "Trunc-to-dio-size failed %d", ret);
cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
return false;
}
}
return true;
}
/*
* Resize the backing object.
*/
static void cachefiles_resize_cookie(struct netfs_cache_resources *cres,
loff_t new_size)
{
struct cachefiles_object *object = cachefiles_cres_object(cres);
struct cachefiles_cache *cache = object->volume->cache;
struct fscache_cookie *cookie = object->cookie;
const struct cred *saved_cred;
struct file *file = cachefiles_cres_file(cres);
loff_t old_size = cookie->object_size;
_enter("%llu->%llu", old_size, new_size);
if (new_size < old_size) {
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_shorten_object(object, file, new_size);
cachefiles_end_secure(cache, saved_cred);
object->cookie->object_size = new_size;
return;
}
/* The file is being expanded. We don't need to do anything
* particularly. cookie->initial_size doesn't change and so the point
* at which we have to download before doesn't change.
*/
cookie->object_size = new_size;
}
/*
* Commit changes to the object as we drop it.
*/
static void cachefiles_commit_object(struct cachefiles_object *object,
struct cachefiles_cache *cache)
{
bool update = false;
if (test_and_clear_bit(FSCACHE_COOKIE_LOCAL_WRITE, &object->cookie->flags))
update = true;
if (test_and_clear_bit(FSCACHE_COOKIE_NEEDS_UPDATE, &object->cookie->flags))
update = true;
if (update)
cachefiles_set_object_xattr(object);
if (test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags))
cachefiles_commit_tmpfile(cache, object);
}
/*
* Finalise and object and close the VFS structs that we have.
*/
static void cachefiles_clean_up_object(struct cachefiles_object *object,
struct cachefiles_cache *cache)
{
if (test_bit(FSCACHE_COOKIE_RETIRED, &object->cookie->flags)) {
if (!test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags)) {
cachefiles_see_object(object, cachefiles_obj_see_clean_delete);
_debug("- inval object OBJ%x", object->debug_id);
cachefiles_delete_object(object, FSCACHE_OBJECT_WAS_RETIRED);
} else {
cachefiles_see_object(object, cachefiles_obj_see_clean_drop_tmp);
_debug("- inval object OBJ%x tmpfile", object->debug_id);
}
} else {
cachefiles_see_object(object, cachefiles_obj_see_clean_commit);
cachefiles_commit_object(object, cache);
}
cachefiles_unmark_inode_in_use(object, object->file);
if (object->file) {
fput(object->file);
object->file = NULL;
}
}
/*
* Withdraw caching for a cookie.
*/
static void cachefiles_withdraw_cookie(struct fscache_cookie *cookie)
{
struct cachefiles_object *object = cookie->cache_priv;
struct cachefiles_cache *cache = object->volume->cache;
const struct cred *saved_cred;
_enter("o=%x", object->debug_id);
cachefiles_see_object(object, cachefiles_obj_see_withdraw_cookie);
if (!list_empty(&object->cache_link)) {
spin_lock(&cache->object_list_lock);
cachefiles_see_object(object, cachefiles_obj_see_withdrawal);
list_del_init(&object->cache_link);
spin_unlock(&cache->object_list_lock);
}
if (object->file) {
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_clean_up_object(object, cache);
cachefiles_end_secure(cache, saved_cred);
}
cookie->cache_priv = NULL;
cachefiles_put_object(object, cachefiles_obj_put_detach);
}
/*
* Invalidate the storage associated with a cookie.
*/
static bool cachefiles_invalidate_cookie(struct fscache_cookie *cookie)
{
struct cachefiles_object *object = cookie->cache_priv;
struct file *new_file, *old_file;
bool old_tmpfile;
_enter("o=%x,[%llu]", object->debug_id, object->cookie->object_size);
old_tmpfile = test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags);
if (!object->file) {
fscache_resume_after_invalidation(cookie);
_leave(" = t [light]");
return true;
}
new_file = cachefiles_create_tmpfile(object);
if (IS_ERR(new_file))
goto failed;
/* Substitute the VFS target */
_debug("sub");
spin_lock(&object->lock);
old_file = object->file;
object->file = new_file;
object->content_info = CACHEFILES_CONTENT_NO_DATA;
set_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags);
set_bit(FSCACHE_COOKIE_NEEDS_UPDATE, &object->cookie->flags);
spin_unlock(&object->lock);
_debug("subbed");
/* Allow I/O to take place again */
fscache_resume_after_invalidation(cookie);
if (old_file) {
if (!old_tmpfile) {
struct cachefiles_volume *volume = object->volume;
struct dentry *fan = volume->fanout[(u8)cookie->key_hash];
inode_lock_nested(d_inode(fan), I_MUTEX_PARENT);
cachefiles_bury_object(volume->cache, object, fan,
old_file->f_path.dentry,
FSCACHE_OBJECT_INVALIDATED);
}
fput(old_file);
}
_leave(" = t");
return true;
failed:
_leave(" = f");
return false;
}
const struct fscache_cache_ops cachefiles_cache_ops = {
.name = "cachefiles",
.alloc_object = cachefiles_alloc_object,
.lookup_object = cachefiles_lookup_object,
.lookup_complete = cachefiles_lookup_complete,
.grab_object = cachefiles_grab_object,
.update_object = cachefiles_update_object,
.invalidate_object = cachefiles_invalidate_object,
.drop_object = cachefiles_drop_object,
.put_object = cachefiles_put_object,
.sync_cache = cachefiles_sync_cache,
.attr_changed = cachefiles_attr_changed,
.read_or_alloc_page = cachefiles_read_or_alloc_page,
.read_or_alloc_pages = cachefiles_read_or_alloc_pages,
.allocate_page = cachefiles_allocate_page,
.allocate_pages = cachefiles_allocate_pages,
.write_page = cachefiles_write_page,
.uncache_page = cachefiles_uncache_page,
.dissociate_pages = cachefiles_dissociate_pages,
.check_consistency = cachefiles_check_consistency,
.begin_read_operation = cachefiles_begin_read_operation,
.acquire_volume = cachefiles_acquire_volume,
.free_volume = cachefiles_free_volume,
.lookup_cookie = cachefiles_lookup_cookie,
.withdraw_cookie = cachefiles_withdraw_cookie,
.invalidate_cookie = cachefiles_invalidate_cookie,
.begin_operation = cachefiles_begin_operation,
.resize_cookie = cachefiles_resize_cookie,
.prepare_to_write = cachefiles_prepare_to_write,
};

286
fs/cachefiles/internal.h
View File

@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/* General netfs cache on cache files internal defs
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
@ -13,58 +13,72 @@
#include <linux/fscache-cache.h>
#include <linux/timer.h>
#include <linux/wait_bit.h>
#include <linux/cred.h>
#include <linux/workqueue.h>
#include <linux/security.h>
#define CACHEFILES_DIO_BLOCK_SIZE 4096
struct cachefiles_cache;
struct cachefiles_object;
extern unsigned cachefiles_debug;
#define CACHEFILES_DEBUG_KENTER 1
#define CACHEFILES_DEBUG_KLEAVE 2
#define CACHEFILES_DEBUG_KDEBUG 4
#define cachefiles_gfp (__GFP_RECLAIM | __GFP_NORETRY | __GFP_NOMEMALLOC)
/*
* node records
*/
struct cachefiles_object {
struct fscache_object fscache; /* fscache handle */
struct cachefiles_lookup_data *lookup_data; /* cached lookup data */
struct dentry *dentry; /* the file/dir representing this object */
struct dentry *backer; /* backing file */
loff_t i_size; /* object size */
unsigned long flags;
#define CACHEFILES_OBJECT_ACTIVE 0 /* T if marked active */
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
uint8_t new; /* T if object new */
spinlock_t work_lock;
struct rb_node active_node; /* link in active tree (dentry is key) */
enum cachefiles_content {
/* These values are saved on disk */
CACHEFILES_CONTENT_NO_DATA = 0, /* No content stored */
CACHEFILES_CONTENT_SINGLE = 1, /* Content is monolithic, all is present */
CACHEFILES_CONTENT_ALL = 2, /* Content is all present, no map */
CACHEFILES_CONTENT_BACKFS_MAP = 3, /* Content is piecemeal, mapped through backing fs */
CACHEFILES_CONTENT_DIRTY = 4, /* Content is dirty (only seen on disk) */
nr__cachefiles_content
};
extern struct kmem_cache *cachefiles_object_jar;
/*
* Cached volume representation.
*/
struct cachefiles_volume {
struct cachefiles_cache *cache;
struct list_head cache_link; /* Link in cache->volumes */
struct fscache_volume *vcookie; /* The netfs's representation */
struct dentry *dentry; /* The volume dentry */
struct dentry *fanout[256]; /* Fanout subdirs */
};
/*
* Backing file state.
*/
struct cachefiles_object {
struct fscache_cookie *cookie; /* Netfs data storage object cookie */
struct cachefiles_volume *volume; /* Cache volume that holds this object */
struct list_head cache_link; /* Link in cache->*_list */
struct file *file; /* The file representing this object */
char *d_name; /* Backing file name */
int debug_id;
spinlock_t lock;
refcount_t ref;
u8 d_name_len; /* Length of filename */
enum cachefiles_content content_info:8; /* Info about content presence */
unsigned long flags;
#define CACHEFILES_OBJECT_USING_TMPFILE 0 /* Have an unlinked tmpfile */
};
/*
* Cache files cache definition
*/
struct cachefiles_cache {
struct fscache_cache cache; /* FS-Cache record */
struct fscache_cache *cache; /* Cache cookie */
struct vfsmount *mnt; /* mountpoint holding the cache */
struct dentry *store; /* Directory into which live objects go */
struct dentry *graveyard; /* directory into which dead objects go */
struct file *cachefilesd; /* manager daemon handle */
struct list_head volumes; /* List of volume objects */
struct list_head object_list; /* List of active objects */
spinlock_t object_list_lock; /* Lock for volumes and object_list */
const struct cred *cache_cred; /* security override for accessing cache */
struct mutex daemon_mutex; /* command serialisation mutex */
wait_queue_head_t daemon_pollwq; /* poll waitqueue for daemon */
struct rb_root active_nodes; /* active nodes (can't be culled) */
rwlock_t active_lock; /* lock for active_nodes */
atomic_t gravecounter; /* graveyard uniquifier */
atomic_t f_released; /* number of objects released lately */
atomic_long_t b_released; /* number of blocks released lately */
atomic_long_t b_writing; /* Number of blocks being written */
unsigned frun_percent; /* when to stop culling (% files) */
unsigned fcull_percent; /* when to start culling (% files) */
unsigned fstop_percent; /* when to stop allocating (% files) */
@ -89,39 +103,20 @@ struct cachefiles_cache {
char *tag; /* cache binding tag */
};
/*
* backing file read tracking
*/
struct cachefiles_one_read {
wait_queue_entry_t monitor; /* link into monitored waitqueue */
struct page *back_page; /* backing file page we're waiting for */
struct page *netfs_page; /* netfs page we're going to fill */
struct fscache_retrieval *op; /* retrieval op covering this */
struct list_head op_link; /* link in op's todo list */
};
/*
* backing file write tracking
*/
struct cachefiles_one_write {
struct page *netfs_page; /* netfs page to copy */
struct cachefiles_object *object;
struct list_head obj_link; /* link in object's lists */
fscache_rw_complete_t end_io_func;
void *context;
};
/*
* auxiliary data xattr buffer
*/
struct cachefiles_xattr {
uint16_t len;
uint8_t type;
uint8_t data[];
};
#include <trace/events/cachefiles.h>
static inline
struct file *cachefiles_cres_file(struct netfs_cache_resources *cres)
{
return cres->cache_priv2;
}
static inline
struct cachefiles_object *cachefiles_cres_object(struct netfs_cache_resources *cres)
{
return fscache_cres_cookie(cres)->cache_priv;
}
/*
* note change of state for daemon
*/
@ -132,74 +127,118 @@ static inline void cachefiles_state_changed(struct cachefiles_cache *cache)
}
/*
* bind.c
* cache.c
*/
extern int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args);
extern void cachefiles_daemon_unbind(struct cachefiles_cache *cache);
extern int cachefiles_add_cache(struct cachefiles_cache *cache);
extern void cachefiles_withdraw_cache(struct cachefiles_cache *cache);
enum cachefiles_has_space_for {
cachefiles_has_space_check,
cachefiles_has_space_for_write,
cachefiles_has_space_for_create,
};
extern int cachefiles_has_space(struct cachefiles_cache *cache,
unsigned fnr, unsigned bnr,
enum cachefiles_has_space_for reason);
/*
* daemon.c
*/
extern const struct file_operations cachefiles_daemon_fops;
extern int cachefiles_has_space(struct cachefiles_cache *cache,
unsigned fnr, unsigned bnr);
/*
* error_inject.c
*/
#ifdef CONFIG_CACHEFILES_ERROR_INJECTION
extern unsigned int cachefiles_error_injection_state;
extern int cachefiles_register_error_injection(void);
extern void cachefiles_unregister_error_injection(void);
#else
#define cachefiles_error_injection_state 0
static inline int cachefiles_register_error_injection(void)
{
return 0;
}
static inline void cachefiles_unregister_error_injection(void)
{
}
#endif
static inline int cachefiles_inject_read_error(void)
{
return cachefiles_error_injection_state & 2 ? -EIO : 0;
}
static inline int cachefiles_inject_write_error(void)
{
return cachefiles_error_injection_state & 2 ? -EIO :
cachefiles_error_injection_state & 1 ? -ENOSPC :
0;
}
static inline int cachefiles_inject_remove_error(void)
{
return cachefiles_error_injection_state & 2 ? -EIO : 0;
}
/*
* interface.c
*/
extern const struct fscache_cache_ops cachefiles_cache_ops;
extern void cachefiles_see_object(struct cachefiles_object *object,
enum cachefiles_obj_ref_trace why);
extern struct cachefiles_object *cachefiles_grab_object(struct cachefiles_object *object,
enum cachefiles_obj_ref_trace why);
extern void cachefiles_put_object(struct cachefiles_object *object,
enum cachefiles_obj_ref_trace why);
void cachefiles_put_object(struct fscache_object *_object,
enum fscache_obj_ref_trace why);
/*
* io.c
*/
extern bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
enum fscache_want_state want_state);
/*
* key.c
*/
extern char *cachefiles_cook_key(const u8 *raw, int keylen, uint8_t type);
extern bool cachefiles_cook_key(struct cachefiles_object *object);
/*
* main.c
*/
extern struct kmem_cache *cachefiles_object_jar;
/*
* namei.c
*/
extern void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
struct cachefiles_object *object,
blkcnt_t i_blocks);
extern int cachefiles_delete_object(struct cachefiles_cache *cache,
struct cachefiles_object *object);
extern int cachefiles_walk_to_object(struct cachefiles_object *parent,
struct cachefiles_object *object,
const char *key,
struct cachefiles_xattr *auxdata);
extern void cachefiles_unmark_inode_in_use(struct cachefiles_object *object,
struct file *file);
extern int cachefiles_bury_object(struct cachefiles_cache *cache,
struct cachefiles_object *object,
struct dentry *dir,
struct dentry *rep,
enum fscache_why_object_killed why);
extern int cachefiles_delete_object(struct cachefiles_object *object,
enum fscache_why_object_killed why);
extern bool cachefiles_look_up_object(struct cachefiles_object *object);
extern struct dentry *cachefiles_get_directory(struct cachefiles_cache *cache,
struct dentry *dir,
const char *name);
const char *name,
bool *_is_new);
extern void cachefiles_put_directory(struct dentry *dir);
extern int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
char *filename);
extern int cachefiles_check_in_use(struct cachefiles_cache *cache,
struct dentry *dir, char *filename);
/*
* rdwr.c
*/
extern int cachefiles_read_or_alloc_page(struct fscache_retrieval *,
struct page *, gfp_t);
extern int cachefiles_read_or_alloc_pages(struct fscache_retrieval *,
struct list_head *, unsigned *,
gfp_t);
extern int cachefiles_allocate_page(struct fscache_retrieval *, struct page *,
gfp_t);
extern int cachefiles_allocate_pages(struct fscache_retrieval *,
struct list_head *, unsigned *, gfp_t);
extern int cachefiles_write_page(struct fscache_storage *, struct page *);
extern void cachefiles_uncache_page(struct fscache_object *, struct page *);
/*
* rdwr2.c
*/
extern int cachefiles_begin_read_operation(struct netfs_read_request *,
struct fscache_retrieval *);
extern struct file *cachefiles_create_tmpfile(struct cachefiles_object *object);
extern bool cachefiles_commit_tmpfile(struct cachefiles_cache *cache,
struct cachefiles_object *object);
/*
* security.c
@ -222,28 +261,32 @@ static inline void cachefiles_end_secure(struct cachefiles_cache *cache,
}
/*
* xattr.c
* volume.c
*/
extern int cachefiles_check_object_type(struct cachefiles_object *object);
extern int cachefiles_set_object_xattr(struct cachefiles_object *object,
struct cachefiles_xattr *auxdata);
extern int cachefiles_update_object_xattr(struct cachefiles_object *object,
struct cachefiles_xattr *auxdata);
extern int cachefiles_check_auxdata(struct cachefiles_object *object);
extern int cachefiles_check_object_xattr(struct cachefiles_object *object,
struct cachefiles_xattr *auxdata);
extern int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
struct dentry *dentry);
void cachefiles_acquire_volume(struct fscache_volume *volume);
void cachefiles_free_volume(struct fscache_volume *volume);
void cachefiles_withdraw_volume(struct cachefiles_volume *volume);
/*
* error handling
* xattr.c
*/
extern int cachefiles_set_object_xattr(struct cachefiles_object *object);
extern int cachefiles_check_auxdata(struct cachefiles_object *object,
struct file *file);
extern int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
struct cachefiles_object *object,
struct dentry *dentry);
extern void cachefiles_prepare_to_write(struct fscache_cookie *cookie);
extern bool cachefiles_set_volume_xattr(struct cachefiles_volume *volume);
extern int cachefiles_check_volume_xattr(struct cachefiles_volume *volume);
/*
* Error handling
*/
#define cachefiles_io_error(___cache, FMT, ...) \
do { \
pr_err("I/O Error: " FMT"\n", ##__VA_ARGS__); \
fscache_io_error(&(___cache)->cache); \
fscache_io_error((___cache)->cache); \
set_bit(CACHEFILES_DEAD, &(___cache)->flags); \
} while (0)
@ -251,15 +294,20 @@ do { \
do { \
struct cachefiles_cache *___cache; \
\
___cache = container_of((object)->fscache.cache, \
struct cachefiles_cache, cache); \
cachefiles_io_error(___cache, FMT, ##__VA_ARGS__); \
___cache = (object)->volume->cache; \
cachefiles_io_error(___cache, FMT " [o=%08x]", ##__VA_ARGS__, \
(object)->debug_id); \
} while (0)
/*
* debug tracing
* Debug tracing
*/
extern unsigned cachefiles_debug;
#define CACHEFILES_DEBUG_KENTER 1
#define CACHEFILES_DEBUG_KLEAVE 2
#define CACHEFILES_DEBUG_KDEBUG 4
#define dbgprintk(FMT, ...) \
printk(KERN_DEBUG "[%-6.6s] "FMT"\n", current->comm, ##__VA_ARGS__)

342
fs/cachefiles/io.c
View File

@ -9,8 +9,9 @@
#include <linux/slab.h>
#include <linux/file.h>
#include <linux/uio.h>
#include <linux/falloc.h>
#include <linux/sched/mm.h>
#include <linux/netfs.h>
#include <trace/events/fscache.h>
#include "internal.h"
struct cachefiles_kiocb {
@ -21,14 +22,18 @@ struct cachefiles_kiocb {
size_t skipped;
size_t len;
};
struct cachefiles_object *object;
netfs_io_terminated_t term_func;
void *term_func_priv;
bool was_async;
unsigned int inval_counter; /* Copy of cookie->inval_counter */
u64 b_writing;
};
static inline void cachefiles_put_kiocb(struct cachefiles_kiocb *ki)
{
if (refcount_dec_and_test(&ki->ki_refcnt)) {
cachefiles_put_object(ki->object, cachefiles_obj_put_ioreq);
fput(ki->iocb.ki_filp);
kfree(ki);
}
@ -40,12 +45,22 @@ static inline void cachefiles_put_kiocb(struct cachefiles_kiocb *ki)
static void cachefiles_read_complete(struct kiocb *iocb, long ret)
{
struct cachefiles_kiocb *ki = container_of(iocb, struct cachefiles_kiocb, iocb);
struct inode *inode = file_inode(ki->iocb.ki_filp);
_enter("%ld", ret);
if (ret < 0)
trace_cachefiles_io_error(ki->object, inode, ret,
cachefiles_trace_read_error);
if (ki->term_func) {
if (ret >= 0)
ret += ki->skipped;
if (ret >= 0) {
if (ki->object->cookie->inval_counter == ki->inval_counter)
ki->skipped += ret;
else
ret = -ESTALE;
}
ki->term_func(ki->term_func_priv, ret, ki->was_async);
}
@ -58,16 +73,24 @@ static void cachefiles_read_complete(struct kiocb *iocb, long ret)
static int cachefiles_read(struct netfs_cache_resources *cres,
loff_t start_pos,
struct iov_iter *iter,
bool seek_data,
enum netfs_read_from_hole read_hole,
netfs_io_terminated_t term_func,
void *term_func_priv)
{
struct cachefiles_object *object;
struct cachefiles_kiocb *ki;
struct file *file = cres->cache_priv2;
struct file *file;
unsigned int old_nofs;
ssize_t ret = -ENOBUFS;
size_t len = iov_iter_count(iter), skipped = 0;
if (!fscache_wait_for_operation(cres, FSCACHE_WANT_READ))
goto presubmission_error;
fscache_count_read();
object = cachefiles_cres_object(cres);
file = cachefiles_cres_file(cres);
_enter("%pD,%li,%llx,%zx/%llx",
file, file_inode(file)->i_ino, start_pos, len,
i_size_read(file_inode(file)));
@ -75,10 +98,12 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
/* If the caller asked us to seek for data before doing the read, then
* we should do that now. If we find a gap, we fill it with zeros.
*/
if (seek_data) {
if (read_hole != NETFS_READ_HOLE_IGNORE) {
loff_t off = start_pos, off2;
off2 = vfs_llseek(file, off, SEEK_DATA);
off2 = cachefiles_inject_read_error();
if (off2 == 0)
off2 = vfs_llseek(file, off, SEEK_DATA);
if (off2 < 0 && off2 >= (loff_t)-MAX_ERRNO && off2 != -ENXIO) {
skipped = 0;
ret = off2;
@ -90,6 +115,10 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
* in the region, so clear the rest of the buffer and
* return success.
*/
ret = -ENODATA;
if (read_hole == NETFS_READ_HOLE_FAIL)
goto presubmission_error;
iov_iter_zero(len, iter);
skipped = len;
ret = 0;
@ -100,7 +129,7 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
iov_iter_zero(skipped, iter);
}
ret = -ENOBUFS;
ret = -ENOMEM;
ki = kzalloc(sizeof(struct cachefiles_kiocb), GFP_KERNEL);
if (!ki)
goto presubmission_error;
@ -112,6 +141,8 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
ki->iocb.ki_hint = ki_hint_validate(file_write_hint(file));
ki->iocb.ki_ioprio = get_current_ioprio();
ki->skipped = skipped;
ki->object = object;
ki->inval_counter = cres->inval_counter;
ki->term_func = term_func;
ki->term_func_priv = term_func_priv;
ki->was_async = true;
@ -120,9 +151,13 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
ki->iocb.ki_complete = cachefiles_read_complete;
get_file(ki->iocb.ki_filp);
cachefiles_grab_object(object, cachefiles_obj_get_ioreq);
trace_cachefiles_read(object, file_inode(file), ki->iocb.ki_pos, len - skipped);
old_nofs = memalloc_nofs_save();
ret = vfs_iocb_iter_read(file, &ki->iocb, iter);
ret = cachefiles_inject_read_error();
if (ret == 0)
ret = vfs_iocb_iter_read(file, &ki->iocb, iter);
memalloc_nofs_restore(old_nofs);
switch (ret) {
case -EIOCBQUEUED:
@ -162,6 +197,7 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
static void cachefiles_write_complete(struct kiocb *iocb, long ret)
{
struct cachefiles_kiocb *ki = container_of(iocb, struct cachefiles_kiocb, iocb);
struct cachefiles_object *object = ki->object;
struct inode *inode = file_inode(ki->iocb.ki_filp);
_enter("%ld", ret);
@ -170,9 +206,14 @@ static void cachefiles_write_complete(struct kiocb *iocb, long ret)
__sb_writers_acquired(inode->i_sb, SB_FREEZE_WRITE);
__sb_end_write(inode->i_sb, SB_FREEZE_WRITE);
if (ret < 0)
trace_cachefiles_io_error(object, inode, ret,
cachefiles_trace_write_error);
atomic_long_sub(ki->b_writing, &object->volume->cache->b_writing);
set_bit(FSCACHE_COOKIE_HAVE_DATA, &object->cookie->flags);
if (ki->term_func)
ki->term_func(ki->term_func_priv, ret, ki->was_async);
cachefiles_put_kiocb(ki);
}
@ -185,17 +226,27 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
netfs_io_terminated_t term_func,
void *term_func_priv)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct cachefiles_kiocb *ki;
struct inode *inode;
struct file *file = cres->cache_priv2;
struct file *file;
unsigned int old_nofs;
ssize_t ret = -ENOBUFS;
size_t len = iov_iter_count(iter);
if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
goto presubmission_error;
fscache_count_write();
object = cachefiles_cres_object(cres);
cache = object->volume->cache;
file = cachefiles_cres_file(cres);
_enter("%pD,%li,%llx,%zx/%llx",
file, file_inode(file)->i_ino, start_pos, len,
i_size_read(file_inode(file)));
ret = -ENOMEM;
ki = kzalloc(sizeof(struct cachefiles_kiocb), GFP_KERNEL);
if (!ki)
goto presubmission_error;
@ -206,14 +257,18 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
ki->iocb.ki_flags = IOCB_DIRECT | IOCB_WRITE;
ki->iocb.ki_hint = ki_hint_validate(file_write_hint(file));
ki->iocb.ki_ioprio = get_current_ioprio();
ki->object = object;
ki->inval_counter = cres->inval_counter;
ki->start = start_pos;
ki->len = len;
ki->term_func = term_func;
ki->term_func_priv = term_func_priv;
ki->was_async = true;
ki->b_writing = (len + (1 << cache->bshift)) >> cache->bshift;
if (ki->term_func)
ki->iocb.ki_complete = cachefiles_write_complete;
atomic_long_add(ki->b_writing, &cache->b_writing);
/* Open-code file_start_write here to grab freeze protection, which
* will be released by another thread in aio_complete_rw(). Fool
@ -225,9 +280,13 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
__sb_writers_release(inode->i_sb, SB_FREEZE_WRITE);
get_file(ki->iocb.ki_filp);
cachefiles_grab_object(object, cachefiles_obj_get_ioreq);
trace_cachefiles_write(object, inode, ki->iocb.ki_pos, len);
old_nofs = memalloc_nofs_save();
ret = vfs_iocb_iter_write(file, &ki->iocb, iter);
ret = cachefiles_inject_write_error();
if (ret == 0)
ret = vfs_iocb_iter_write(file, &ki->iocb, iter);
memalloc_nofs_restore(old_nofs);
switch (ret) {
case -EIOCBQUEUED:
@ -257,8 +316,8 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
presubmission_error:
if (term_func)
term_func(term_func_priv, -ENOMEM, false);
return -ENOMEM;
term_func(term_func_priv, ret, false);
return ret;
}
/*
@ -268,47 +327,82 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
static enum netfs_read_source cachefiles_prepare_read(struct netfs_read_subrequest *subreq,
loff_t i_size)
{
struct fscache_retrieval *op = subreq->rreq->cache_resources.cache_priv;
enum cachefiles_prepare_read_trace why;
struct netfs_read_request *rreq = subreq->rreq;
struct netfs_cache_resources *cres = &rreq->cache_resources;
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct fscache_cookie *cookie = fscache_cres_cookie(cres);
const struct cred *saved_cred;
struct file *file = subreq->rreq->cache_resources.cache_priv2;
struct file *file = cachefiles_cres_file(cres);
enum netfs_read_source ret = NETFS_DOWNLOAD_FROM_SERVER;
loff_t off, to;
ino_t ino = file ? file_inode(file)->i_ino : 0;
_enter("%zx @%llx/%llx", subreq->len, subreq->start, i_size);
object = container_of(op->op.object,
struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
if (!file)
goto cache_fail_nosec;
if (subreq->start >= i_size)
return NETFS_FILL_WITH_ZEROES;
cachefiles_begin_secure(cache, &saved_cred);
off = vfs_llseek(file, subreq->start, SEEK_DATA);
if (off < 0 && off >= (loff_t)-MAX_ERRNO) {
if (off == (loff_t)-ENXIO)
goto download_and_store;
goto cache_fail;
if (subreq->start >= i_size) {
ret = NETFS_FILL_WITH_ZEROES;
why = cachefiles_trace_read_after_eof;
goto out_no_object;
}
if (off >= subreq->start + subreq->len)
if (test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) {
__set_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
why = cachefiles_trace_read_no_data;
goto out_no_object;
}
/* The object and the file may be being created in the background. */
if (!file) {
why = cachefiles_trace_read_no_file;
if (!fscache_wait_for_operation(cres, FSCACHE_WANT_READ))
goto out_no_object;
file = cachefiles_cres_file(cres);
if (!file)
goto out_no_object;
ino = file_inode(file)->i_ino;
}
object = cachefiles_cres_object(cres);
cache = object->volume->cache;
cachefiles_begin_secure(cache, &saved_cred);
off = cachefiles_inject_read_error();
if (off == 0)
off = vfs_llseek(file, subreq->start, SEEK_DATA);
if (off < 0 && off >= (loff_t)-MAX_ERRNO) {
if (off == (loff_t)-ENXIO) {
why = cachefiles_trace_read_seek_nxio;
goto download_and_store;
}
trace_cachefiles_io_error(object, file_inode(file), off,
cachefiles_trace_seek_error);
why = cachefiles_trace_read_seek_error;
goto out;
}
if (off >= subreq->start + subreq->len) {
why = cachefiles_trace_read_found_hole;
goto download_and_store;
}
if (off > subreq->start) {
off = round_up(off, cache->bsize);
subreq->len = off - subreq->start;
why = cachefiles_trace_read_found_part;
goto download_and_store;
}
to = vfs_llseek(file, subreq->start, SEEK_HOLE);
if (to < 0 && to >= (loff_t)-MAX_ERRNO)
goto cache_fail;
to = cachefiles_inject_read_error();
if (to == 0)
to = vfs_llseek(file, subreq->start, SEEK_HOLE);
if (to < 0 && to >= (loff_t)-MAX_ERRNO) {
trace_cachefiles_io_error(object, file_inode(file), to,
cachefiles_trace_seek_error);
why = cachefiles_trace_read_seek_error;
goto out;
}
if (to < subreq->start + subreq->len) {
if (subreq->start + subreq->len >= i_size)
@ -318,32 +412,119 @@ static enum netfs_read_source cachefiles_prepare_read(struct netfs_read_subreque
subreq->len = to - subreq->start;
}
cachefiles_end_secure(cache, saved_cred);
return NETFS_READ_FROM_CACHE;
why = cachefiles_trace_read_have_data;
ret = NETFS_READ_FROM_CACHE;
goto out;
download_and_store:
if (cachefiles_has_space(cache, 0, (subreq->len + PAGE_SIZE - 1) / PAGE_SIZE) == 0)
__set_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
cache_fail:
__set_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
out:
cachefiles_end_secure(cache, saved_cred);
cache_fail_nosec:
return NETFS_DOWNLOAD_FROM_SERVER;
out_no_object:
trace_cachefiles_prep_read(subreq, ret, why, ino);
return ret;
}
/*
* Prepare for a write to occur.
*/
static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
loff_t *_start, size_t *_len, loff_t i_size)
static int __cachefiles_prepare_write(struct netfs_cache_resources *cres,
loff_t *_start, size_t *_len, loff_t i_size,
bool no_space_allocated_yet)
{
loff_t start = *_start;
struct cachefiles_object *object = cachefiles_cres_object(cres);
struct cachefiles_cache *cache = object->volume->cache;
struct file *file = cachefiles_cres_file(cres);
loff_t start = *_start, pos;
size_t len = *_len, down;
int ret;
/* Round to DIO size */
down = start - round_down(start, PAGE_SIZE);
*_start = start - down;
*_len = round_up(down + len, PAGE_SIZE);
return 0;
/* We need to work out whether there's sufficient disk space to perform
* the write - but we can skip that check if we have space already
* allocated.
*/
if (no_space_allocated_yet)
goto check_space;
pos = cachefiles_inject_read_error();
if (pos == 0)
pos = vfs_llseek(file, *_start, SEEK_DATA);
if (pos < 0 && pos >= (loff_t)-MAX_ERRNO) {
if (pos == -ENXIO)
goto check_space; /* Unallocated tail */
trace_cachefiles_io_error(object, file_inode(file), pos,
cachefiles_trace_seek_error);
return pos;
}
if ((u64)pos >= (u64)*_start + *_len)
goto check_space; /* Unallocated region */
/* We have a block that's at least partially filled - if we're low on
* space, we need to see if it's fully allocated. If it's not, we may
* want to cull it.
*/
if (cachefiles_has_space(cache, 0, *_len / PAGE_SIZE,
cachefiles_has_space_check) == 0)
return 0; /* Enough space to simply overwrite the whole block */
pos = cachefiles_inject_read_error();
if (pos == 0)
pos = vfs_llseek(file, *_start, SEEK_HOLE);
if (pos < 0 && pos >= (loff_t)-MAX_ERRNO) {
trace_cachefiles_io_error(object, file_inode(file), pos,
cachefiles_trace_seek_error);
return pos;
}
if ((u64)pos >= (u64)*_start + *_len)
return 0; /* Fully allocated */
/* Partially allocated, but insufficient space: cull. */
fscache_count_no_write_space();
ret = cachefiles_inject_remove_error();
if (ret == 0)
ret = vfs_fallocate(file, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
*_start, *_len);
if (ret < 0) {
trace_cachefiles_io_error(object, file_inode(file), ret,
cachefiles_trace_fallocate_error);
cachefiles_io_error_obj(object,
"CacheFiles: fallocate failed (%d)\n", ret);
ret = -EIO;
}
return ret;
check_space:
return cachefiles_has_space(cache, 0, *_len / PAGE_SIZE,
cachefiles_has_space_for_write);
}
static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
loff_t *_start, size_t *_len, loff_t i_size,
bool no_space_allocated_yet)
{
struct cachefiles_object *object = cachefiles_cres_object(cres);
struct cachefiles_cache *cache = object->volume->cache;
const struct cred *saved_cred;
int ret;
if (!cachefiles_cres_file(cres)) {
if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
return -ENOBUFS;
if (!cachefiles_cres_file(cres))
return -ENOBUFS;
}
cachefiles_begin_secure(cache, &saved_cred);
ret = __cachefiles_prepare_write(cres, _start, _len, i_size,
no_space_allocated_yet);
cachefiles_end_secure(cache, saved_cred);
return ret;
}
/*
@ -351,19 +532,11 @@ static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
*/
static void cachefiles_end_operation(struct netfs_cache_resources *cres)
{
struct fscache_retrieval *op = cres->cache_priv;
struct file *file = cres->cache_priv2;
_enter("");
struct file *file = cachefiles_cres_file(cres);
if (file)
fput(file);
if (op) {
fscache_op_complete(&op->op, false);
fscache_put_retrieval(op);
}
_leave("");
fscache_end_cookie_access(fscache_cres_cookie(cres), fscache_access_io_end);
}
static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
@ -377,44 +550,25 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
/*
* Open the cache file when beginning a cache operation.
*/
int cachefiles_begin_read_operation(struct netfs_read_request *rreq,
struct fscache_retrieval *op)
bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
enum fscache_want_state want_state)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct path path;
struct file *file;
struct cachefiles_object *object = cachefiles_cres_object(cres);
_enter("");
object = container_of(op->op.object,
struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
path.mnt = cache->mnt;
path.dentry = object->backer;
file = open_with_fake_path(&path, O_RDWR | O_LARGEFILE | O_DIRECT,
d_inode(object->backer), cache->cache_cred);
if (IS_ERR(file))
return PTR_ERR(file);
if (!S_ISREG(file_inode(file)->i_mode))
goto error_file;
if (unlikely(!file->f_op->read_iter) ||
unlikely(!file->f_op->write_iter)) {
pr_notice("Cache does not support read_iter and write_iter\n");
goto error_file;
if (!cachefiles_cres_file(cres)) {
cres->ops = &cachefiles_netfs_cache_ops;
if (object->file) {
spin_lock(&object->lock);
if (!cres->cache_priv2 && object->file)
cres->cache_priv2 = get_file(object->file);
spin_unlock(&object->lock);
}
}
fscache_get_retrieval(op);
rreq->cache_resources.cache_priv = op;
rreq->cache_resources.cache_priv2 = file;
rreq->cache_resources.ops = &cachefiles_netfs_cache_ops;
rreq->cache_resources.debug_id = object->fscache.debug_id;
_leave("");
return 0;
if (!cachefiles_cres_file(cres) && want_state != FSCACHE_WANT_PARAMS) {
pr_err("failed to get cres->file\n");
return false;
}
error_file:
fput(file);
return -EIO;
return true;
}

195
fs/cachefiles/key.c
View File

@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Key to pathname encoder
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
@ -22,134 +22,117 @@ static const char cachefiles_filecharmap[256] = {
[48 ... 127] = 1, /* '0' -> '~' */
};
static inline unsigned int how_many_hex_digits(unsigned int x)
{
return x ? round_up(ilog2(x) + 1, 4) / 4 : 0;
}
/*
* turn the raw key into something cooked
* - the raw key should include the length in the two bytes at the front
* - the key may be up to 514 bytes in length (including the length word)
* - the key may be up to NAME_MAX in length (including the length word)
* - "base64" encode the strange keys, mapping 3 bytes of raw to four of
* cooked
* - need to cut the cooked key into 252 char lengths (189 raw bytes)
*/
char *cachefiles_cook_key(const u8 *raw, int keylen, uint8_t type)
bool cachefiles_cook_key(struct cachefiles_object *object)
{
unsigned char csum, ch;
unsigned int acc;
char *key;
int loop, len, max, seg, mark, print;
const u8 *key = fscache_get_key(object->cookie), *kend;
unsigned char ch;
unsigned int acc, i, n, nle, nbe, keylen = object->cookie->key_len;
unsigned int b64len, len, print, pad;
char *name, sep;
_enter(",%d", keylen);
_enter(",%u,%*phN", keylen, keylen, key);
BUG_ON(keylen < 2 || keylen > 514);
BUG_ON(keylen > NAME_MAX - 3);
csum = raw[0] + raw[1];
print = 1;
for (loop = 2; loop < keylen; loop++) {
ch = raw[loop];
csum += ch;
for (i = 0; i < keylen; i++) {
ch = key[i];
print &= cachefiles_filecharmap[ch];
}
/* If the path is usable ASCII, then we render it directly */
if (print) {
/* if the path is usable ASCII, then we render it directly */
max = keylen - 2;
max += 2; /* two base64'd length chars on the front */
max += 5; /* @checksum/M */
max += 3 * 2; /* maximum number of segment dividers (".../M")
* is ((514 + 251) / 252) = 3
*/
max += 1; /* NUL on end */
} else {
/* calculate the maximum length of the cooked key */
keylen = (keylen + 2) / 3;
len = 1 + keylen;
name = kmalloc(len + 1, GFP_KERNEL);
if (!name)
return false;
max = keylen * 4;
max += 5; /* @checksum/M */
max += 3 * 2; /* maximum number of segment dividers (".../M")
* is ((514 + 188) / 189) = 3
*/
max += 1; /* NUL on end */
name[0] = 'D'; /* Data object type, string encoding */
memcpy(name + 1, key, keylen);
goto success;
}
max += 1; /* 2nd NUL on end */
/* See if it makes sense to encode it as "hex,hex,hex" for each 32-bit
* chunk. We rely on the key having been padded out to a whole number
* of 32-bit words.
*/
n = round_up(keylen, 4);
nbe = nle = 0;
for (i = 0; i < n; i += 4) {
u32 be = be32_to_cpu(*(__be32 *)(key + i));
u32 le = le32_to_cpu(*(__le32 *)(key + i));
_debug("max: %d", max);
nbe += 1 + how_many_hex_digits(be);
nle += 1 + how_many_hex_digits(le);
}
key = kmalloc(max, cachefiles_gfp);
if (!key)
return NULL;
b64len = DIV_ROUND_UP(keylen, 3);
pad = b64len * 3 - keylen;
b64len = 2 + b64len * 4; /* Length if we base64-encode it */
_debug("len=%u nbe=%u nle=%u b64=%u", keylen, nbe, nle, b64len);
if (nbe < b64len || nle < b64len) {
unsigned int nlen = min(nbe, nle) + 1;
name = kmalloc(nlen, GFP_KERNEL);
if (!name)
return false;
sep = (nbe <= nle) ? 'S' : 'T'; /* Encoding indicator */
len = 0;
for (i = 0; i < n; i += 4) {
u32 x;
if (nbe <= nle)
x = be32_to_cpu(*(__be32 *)(key + i));
else
x = le32_to_cpu(*(__le32 *)(key + i));
name[len++] = sep;
if (x != 0)
len += snprintf(name + len, nlen - len, "%x", x);
sep = ',';
}
goto success;
}
len = 0;
/* We need to base64-encode it */
name = kmalloc(b64len + 1, GFP_KERNEL);
if (!name)
return false;
/* build the cooked key */
sprintf(key, "@%02x%c+", (unsigned) csum, 0);
len = 5;
mark = len - 1;
name[0] = 'E';
name[1] = '0' + pad;
len = 2;
kend = key + keylen;
do {
acc = *key++;
if (key < kend) {
acc |= *key++ << 8;
if (key < kend)
acc |= *key++ << 16;
}
if (print) {
acc = *(uint16_t *) raw;
raw += 2;
key[len + 1] = cachefiles_charmap[acc & 63];
name[len++] = cachefiles_charmap[acc & 63];
acc >>= 6;
key[len] = cachefiles_charmap[acc & 63];
len += 2;
name[len++] = cachefiles_charmap[acc & 63];
acc >>= 6;
name[len++] = cachefiles_charmap[acc & 63];
acc >>= 6;
name[len++] = cachefiles_charmap[acc & 63];
} while (key < kend);
seg = 250;
for (loop = keylen; loop > 0; loop--) {
if (seg <= 0) {
key[len++] = '\0';
mark = len;
key[len++] = '+';
seg = 252;
}
key[len++] = *raw++;
ASSERT(len < max);
}
switch (type) {
case FSCACHE_COOKIE_TYPE_INDEX: type = 'I'; break;
case FSCACHE_COOKIE_TYPE_DATAFILE: type = 'D'; break;
default: type = 'S'; break;
}
} else {
seg = 252;
for (loop = keylen; loop > 0; loop--) {
if (seg <= 0) {
key[len++] = '\0';
mark = len;
key[len++] = '+';
seg = 252;
}
acc = *raw++;
acc |= *raw++ << 8;
acc |= *raw++ << 16;
_debug("acc: %06x", acc);
key[len++] = cachefiles_charmap[acc & 63];
acc >>= 6;
key[len++] = cachefiles_charmap[acc & 63];
acc >>= 6;
key[len++] = cachefiles_charmap[acc & 63];
acc >>= 6;
key[len++] = cachefiles_charmap[acc & 63];
ASSERT(len < max);
}
switch (type) {
case FSCACHE_COOKIE_TYPE_INDEX: type = 'J'; break;
case FSCACHE_COOKIE_TYPE_DATAFILE: type = 'E'; break;
default: type = 'T'; break;
}
}
key[mark] = type;
key[len++] = 0;
key[len] = 0;
_leave(" = %s %d", key, len);
return key;
success:
name[len] = 0;
object->d_name = name;
object->d_name_len = len;
_leave(" = %s", object->d_name);
return true;
}

22
fs/cachefiles/main.c
View File

@ -2,7 +2,7 @@
/* Network filesystem caching backend to use cache files on a premounted
* filesystem
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
@ -18,6 +18,8 @@
#include <linux/statfs.h>
#include <linux/sysctl.h>
#include <linux/miscdevice.h>
#include <linux/netfs.h>
#include <trace/events/netfs.h>
#define CREATE_TRACE_POINTS
#include "internal.h"
@ -37,14 +39,6 @@ static struct miscdevice cachefiles_dev = {
.fops = &cachefiles_daemon_fops,
};
static void cachefiles_object_init_once(void *_object)
{
struct cachefiles_object *object = _object;
memset(object, 0, sizeof(*object));
spin_lock_init(&object->work_lock);
}
/*
* initialise the fs caching module
*/
@ -52,6 +46,9 @@ static int __init cachefiles_init(void)
{
int ret;
ret = cachefiles_register_error_injection();
if (ret < 0)
goto error_einj;
ret = misc_register(&cachefiles_dev);
if (ret < 0)
goto error_dev;
@ -61,9 +58,7 @@ static int __init cachefiles_init(void)
cachefiles_object_jar =
kmem_cache_create("cachefiles_object_jar",
sizeof(struct cachefiles_object),
0,
SLAB_HWCACHE_ALIGN,
cachefiles_object_init_once);
0, SLAB_HWCACHE_ALIGN, NULL);
if (!cachefiles_object_jar) {
pr_notice("Failed to allocate an object jar\n");
goto error_object_jar;
@ -75,6 +70,8 @@ static int __init cachefiles_init(void)
error_object_jar:
misc_deregister(&cachefiles_dev);
error_dev:
cachefiles_unregister_error_injection();
error_einj:
pr_err("failed to register: %d\n", ret);
return ret;
}
@ -90,6 +87,7 @@ static void __exit cachefiles_exit(void)
kmem_cache_destroy(cachefiles_object_jar);
misc_deregister(&cachefiles_dev);
cachefiles_unregister_error_injection();
}
module_exit(cachefiles_exit);

1343
fs/cachefiles/namei.c
View File

File diff suppressed because it is too large Load Diff

972
fs/cachefiles/rdwr.c
View File

@ -1,972 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Storage object read/write
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/mount.h>
#include <linux/slab.h>
#include <linux/file.h>
#include <linux/swap.h>
#include "internal.h"
/*
* detect wake up events generated by the unlocking of pages in which we're
* interested
* - we use this to detect read completion of backing pages
* - the caller holds the waitqueue lock
*/
static int cachefiles_read_waiter(wait_queue_entry_t *wait, unsigned mode,
int sync, void *_key)
{
struct cachefiles_one_read *monitor =
container_of(wait, struct cachefiles_one_read, monitor);
struct cachefiles_object *object;
struct fscache_retrieval *op = monitor->op;
struct wait_page_key *key = _key;
struct folio *folio = wait->private;
ASSERT(key);
_enter("{%lu},%u,%d,{%p,%u}",
monitor->netfs_page->index, mode, sync,
key->folio, key->bit_nr);
if (key->folio != folio || key->bit_nr != PG_locked)
return 0;
_debug("--- monitor %p %lx ---", folio, folio->flags);
if (!folio_test_uptodate(folio) && !folio_test_error(folio)) {
/* unlocked, not uptodate and not erronous? */
_debug("page probably truncated");
}
/* remove from the waitqueue */
list_del(&wait->entry);
/* move onto the action list and queue for FS-Cache thread pool */
ASSERT(op);
/* We need to temporarily bump the usage count as we don't own a ref
* here otherwise cachefiles_read_copier() may free the op between the
* monitor being enqueued on the op->to_do list and the op getting
* enqueued on the work queue.
*/
fscache_get_retrieval(op);
object = container_of(op->op.object, struct cachefiles_object, fscache);
spin_lock(&object->work_lock);
list_add_tail(&monitor->op_link, &op->to_do);
fscache_enqueue_retrieval(op);
spin_unlock(&object->work_lock);
fscache_put_retrieval(op);
return 0;
}
/*
* handle a probably truncated page
* - check to see if the page is still relevant and reissue the read if
* possible
* - return -EIO on error, -ENODATA if the page is gone, -EINPROGRESS if we
* must wait again and 0 if successful
*/
static int cachefiles_read_reissue(struct cachefiles_object *object,
struct cachefiles_one_read *monitor)
{
struct address_space *bmapping = d_backing_inode(object->backer)->i_mapping;
struct page *backpage = monitor->back_page, *backpage2;
int ret;
_enter("{ino=%lx},{%lx,%lx}",
d_backing_inode(object->backer)->i_ino,
backpage->index, backpage->flags);
/* skip if the page was truncated away completely */
if (backpage->mapping != bmapping) {
_leave(" = -ENODATA [mapping]");
return -ENODATA;
}
backpage2 = find_get_page(bmapping, backpage->index);
if (!backpage2) {
_leave(" = -ENODATA [gone]");
return -ENODATA;
}
if (backpage != backpage2) {
put_page(backpage2);
_leave(" = -ENODATA [different]");
return -ENODATA;
}
/* the page is still there and we already have a ref on it, so we don't
* need a second */
put_page(backpage2);
INIT_LIST_HEAD(&monitor->op_link);
folio_add_wait_queue(page_folio(backpage), &monitor->monitor);
if (trylock_page(backpage)) {
ret = -EIO;
if (PageError(backpage))
goto unlock_discard;
ret = 0;
if (PageUptodate(backpage))
goto unlock_discard;
_debug("reissue read");
ret = bmapping->a_ops->readpage(NULL, backpage);
if (ret < 0)
goto discard;
}
/* but the page may have been read before the monitor was installed, so
* the monitor may miss the event - so we have to ensure that we do get
* one in such a case */
if (trylock_page(backpage)) {
_debug("jumpstart %p {%lx}", backpage, backpage->flags);
unlock_page(backpage);
}
/* it'll reappear on the todo list */
_leave(" = -EINPROGRESS");
return -EINPROGRESS;
unlock_discard:
unlock_page(backpage);
discard:
spin_lock_irq(&object->work_lock);
list_del(&monitor->op_link);
spin_unlock_irq(&object->work_lock);
_leave(" = %d", ret);
return ret;
}
/*
* copy data from backing pages to netfs pages to complete a read operation
* - driven by FS-Cache's thread pool
*/
static void cachefiles_read_copier(struct fscache_operation *_op)
{
struct cachefiles_one_read *monitor;
struct cachefiles_object *object;
struct fscache_retrieval *op;
int error, max;
op = container_of(_op, struct fscache_retrieval, op);
object = container_of(op->op.object,
struct cachefiles_object, fscache);
_enter("{ino=%lu}", d_backing_inode(object->backer)->i_ino);
max = 8;
spin_lock_irq(&object->work_lock);
while (!list_empty(&op->to_do)) {
monitor = list_entry(op->to_do.next,
struct cachefiles_one_read, op_link);
list_del(&monitor->op_link);
spin_unlock_irq(&object->work_lock);
_debug("- copy {%lu}", monitor->back_page->index);
recheck:
if (test_bit(FSCACHE_COOKIE_INVALIDATING,
&object->fscache.cookie->flags)) {
error = -ESTALE;
} else if (PageUptodate(monitor->back_page)) {
copy_highpage(monitor->netfs_page, monitor->back_page);
fscache_mark_page_cached(monitor->op,
monitor->netfs_page);
error = 0;
} else if (!PageError(monitor->back_page)) {
/* the page has probably been truncated */
error = cachefiles_read_reissue(object, monitor);
if (error == -EINPROGRESS)
goto next;
goto recheck;
} else {
cachefiles_io_error_obj(
object,
"Readpage failed on backing file %lx",
(unsigned long) monitor->back_page->flags);
error = -EIO;
}
put_page(monitor->back_page);
fscache_end_io(op, monitor->netfs_page, error);
put_page(monitor->netfs_page);
fscache_retrieval_complete(op, 1);
fscache_put_retrieval(op);
kfree(monitor);
next:
/* let the thread pool have some air occasionally */
max--;
if (max < 0 || need_resched()) {
if (!list_empty(&op->to_do))
fscache_enqueue_retrieval(op);
_leave(" [maxed out]");
return;
}
spin_lock_irq(&object->work_lock);
}
spin_unlock_irq(&object->work_lock);
_leave("");
}
/*
* read the corresponding page to the given set from the backing file
* - an uncertain page is simply discarded, to be tried again another time
*/
static int cachefiles_read_backing_file_one(struct cachefiles_object *object,
struct fscache_retrieval *op,
struct page *netpage)
{
struct cachefiles_one_read *monitor;
struct address_space *bmapping;
struct page *newpage, *backpage;
int ret;
_enter("");
_debug("read back %p{%lu,%d}",
netpage, netpage->index, page_count(netpage));
monitor = kzalloc(sizeof(*monitor), cachefiles_gfp);
if (!monitor)
goto nomem;
monitor->netfs_page = netpage;
monitor->op = fscache_get_retrieval(op);
init_waitqueue_func_entry(&monitor->monitor, cachefiles_read_waiter);
/* attempt to get hold of the backing page */
bmapping = d_backing_inode(object->backer)->i_mapping;
newpage = NULL;
for (;;) {
backpage = find_get_page(bmapping, netpage->index);
if (backpage)
goto backing_page_already_present;
if (!newpage) {
newpage = __page_cache_alloc(cachefiles_gfp);
if (!newpage)
goto nomem_monitor;
}
ret = add_to_page_cache_lru(newpage, bmapping,
netpage->index, cachefiles_gfp);
if (ret == 0)
goto installed_new_backing_page;
if (ret != -EEXIST)
goto nomem_page;
}
/* we've installed a new backing page, so now we need to start
* it reading */
installed_new_backing_page:
_debug("- new %p", newpage);
backpage = newpage;
newpage = NULL;
read_backing_page:
ret = bmapping->a_ops->readpage(NULL, backpage);
if (ret < 0)
goto read_error;
/* set the monitor to transfer the data across */
monitor_backing_page:
_debug("- monitor add");
/* install the monitor */
get_page(monitor->netfs_page);
get_page(backpage);
monitor->back_page = backpage;
monitor->monitor.private = backpage;
folio_add_wait_queue(page_folio(backpage), &monitor->monitor);
monitor = NULL;
/* but the page may have been read before the monitor was installed, so
* the monitor may miss the event - so we have to ensure that we do get
* one in such a case */
if (trylock_page(backpage)) {
_debug("jumpstart %p {%lx}", backpage, backpage->flags);
unlock_page(backpage);
}
goto success;
/* if the backing page is already present, it can be in one of
* three states: read in progress, read failed or read okay */
backing_page_already_present:
_debug("- present");
if (newpage) {
put_page(newpage);
newpage = NULL;
}
if (PageError(backpage))
goto io_error;
if (PageUptodate(backpage))
goto backing_page_already_uptodate;
if (!trylock_page(backpage))
goto monitor_backing_page;
_debug("read %p {%lx}", backpage, backpage->flags);
goto read_backing_page;
/* the backing page is already up to date, attach the netfs
* page to the pagecache and LRU and copy the data across */
backing_page_already_uptodate:
_debug("- uptodate");
fscache_mark_page_cached(op, netpage);
copy_highpage(netpage, backpage);
fscache_end_io(op, netpage, 0);
fscache_retrieval_complete(op, 1);
success:
_debug("success");
ret = 0;
out:
if (backpage)
put_page(backpage);
if (monitor) {
fscache_put_retrieval(monitor->op);
kfree(monitor);
}
_leave(" = %d", ret);
return ret;
read_error:
_debug("read error %d", ret);
if (ret == -ENOMEM) {
fscache_retrieval_complete(op, 1);
goto out;
}
io_error:
cachefiles_io_error_obj(object, "Page read error on backing file");
fscache_retrieval_complete(op, 1);
ret = -ENOBUFS;
goto out;
nomem_page:
put_page(newpage);
nomem_monitor:
fscache_put_retrieval(monitor->op);
kfree(monitor);
nomem:
fscache_retrieval_complete(op, 1);
_leave(" = -ENOMEM");
return -ENOMEM;
}
/*
* read a page from the cache or allocate a block in which to store it
* - cache withdrawal is prevented by the caller
* - returns -EINTR if interrupted
* - returns -ENOMEM if ran out of memory
* - returns -ENOBUFS if no buffers can be made available
* - returns -ENOBUFS if page is beyond EOF
* - if the page is backed by a block in the cache:
* - a read will be started which will call the callback on completion
* - 0 will be returned
* - else if the page is unbacked:
* - the metadata will be retained
* - -ENODATA will be returned
*/
int cachefiles_read_or_alloc_page(struct fscache_retrieval *op,
struct page *page,
gfp_t gfp)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct inode *inode;
sector_t block;
unsigned shift;
int ret, ret2;
object = container_of(op->op.object,
struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
_enter("{%p},{%lx},,,", object, page->index);
if (!object->backer)
goto enobufs;
inode = d_backing_inode(object->backer);
ASSERT(S_ISREG(inode->i_mode));
/* calculate the shift required to use bmap */
shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits;
op->op.flags &= FSCACHE_OP_KEEP_FLAGS;
op->op.flags |= FSCACHE_OP_ASYNC;
op->op.processor = cachefiles_read_copier;
/* we assume the absence or presence of the first block is a good
* enough indication for the page as a whole
* - TODO: don't use bmap() for this as it is _not_ actually good
* enough for this as it doesn't indicate errors, but it's all we've
* got for the moment
*/
block = page->index;
block <<= shift;
ret2 = bmap(inode, &block);
ASSERT(ret2 == 0);
_debug("%llx -> %llx",
(unsigned long long) (page->index << shift),
(unsigned long long) block);
if (block) {
/* submit the apparently valid page to the backing fs to be
* read from disk */
ret = cachefiles_read_backing_file_one(object, op, page);
} else if (cachefiles_has_space(cache, 0, 1) == 0) {
/* there's space in the cache we can use */
fscache_mark_page_cached(op, page);
fscache_retrieval_complete(op, 1);
ret = -ENODATA;
} else {
goto enobufs;
}
_leave(" = %d", ret);
return ret;
enobufs:
fscache_retrieval_complete(op, 1);
_leave(" = -ENOBUFS");
return -ENOBUFS;
}
/*
* read the corresponding pages to the given set from the backing file
* - any uncertain pages are simply discarded, to be tried again another time
*/
static int cachefiles_read_backing_file(struct cachefiles_object *object,
struct fscache_retrieval *op,
struct list_head *list)
{
struct cachefiles_one_read *monitor = NULL;
struct address_space *bmapping = d_backing_inode(object->backer)->i_mapping;
struct page *newpage = NULL, *netpage, *_n, *backpage = NULL;
int ret = 0;
_enter("");
list_for_each_entry_safe(netpage, _n, list, lru) {
list_del(&netpage->lru);
_debug("read back %p{%lu,%d}",
netpage, netpage->index, page_count(netpage));
if (!monitor) {
monitor = kzalloc(sizeof(*monitor), cachefiles_gfp);
if (!monitor)
goto nomem;
monitor->op = fscache_get_retrieval(op);
init_waitqueue_func_entry(&monitor->monitor,
cachefiles_read_waiter);
}
for (;;) {
backpage = find_get_page(bmapping, netpage->index);
if (backpage)
goto backing_page_already_present;
if (!newpage) {
newpage = __page_cache_alloc(cachefiles_gfp);
if (!newpage)
goto nomem;
}
ret = add_to_page_cache_lru(newpage, bmapping,
netpage->index,
cachefiles_gfp);
if (ret == 0)
goto installed_new_backing_page;
if (ret != -EEXIST)
goto nomem;
}
/* we've installed a new backing page, so now we need
* to start it reading */
installed_new_backing_page:
_debug("- new %p", newpage);
backpage = newpage;
newpage = NULL;
reread_backing_page:
ret = bmapping->a_ops->readpage(NULL, backpage);
if (ret < 0)
goto read_error;
/* add the netfs page to the pagecache and LRU, and set the
* monitor to transfer the data across */
monitor_backing_page:
_debug("- monitor add");
ret = add_to_page_cache_lru(netpage, op->mapping,
netpage->index, cachefiles_gfp);
if (ret < 0) {
if (ret == -EEXIST) {
put_page(backpage);
backpage = NULL;
put_page(netpage);
netpage = NULL;
fscache_retrieval_complete(op, 1);
continue;
}
goto nomem;
}
/* install a monitor */
get_page(netpage);
monitor->netfs_page = netpage;
get_page(backpage);
monitor->back_page = backpage;
monitor->monitor.private = backpage;
folio_add_wait_queue(page_folio(backpage), &monitor->monitor);
monitor = NULL;
/* but the page may have been read before the monitor was
* installed, so the monitor may miss the event - so we have to
* ensure that we do get one in such a case */
if (trylock_page(backpage)) {
_debug("2unlock %p {%lx}", backpage, backpage->flags);
unlock_page(backpage);
}
put_page(backpage);
backpage = NULL;
put_page(netpage);
netpage = NULL;
continue;
/* if the backing page is already present, it can be in one of
* three states: read in progress, read failed or read okay */
backing_page_already_present:
_debug("- present %p", backpage);
if (PageError(backpage))
goto io_error;
if (PageUptodate(backpage))
goto backing_page_already_uptodate;
_debug("- not ready %p{%lx}", backpage, backpage->flags);
if (!trylock_page(backpage))
goto monitor_backing_page;
if (PageError(backpage)) {
_debug("error %lx", backpage->flags);
unlock_page(backpage);
goto io_error;
}
if (PageUptodate(backpage))
goto backing_page_already_uptodate_unlock;
/* we've locked a page that's neither up to date nor erroneous,
* so we need to attempt to read it again */
goto reread_backing_page;
/* the backing page is already up to date, attach the netfs
* page to the pagecache and LRU and copy the data across */
backing_page_already_uptodate_unlock:
_debug("uptodate %lx", backpage->flags);
unlock_page(backpage);
backing_page_already_uptodate:
_debug("- uptodate");
ret = add_to_page_cache_lru(netpage, op->mapping,
netpage->index, cachefiles_gfp);
if (ret < 0) {
if (ret == -EEXIST) {
put_page(backpage);
backpage = NULL;
put_page(netpage);
netpage = NULL;
fscache_retrieval_complete(op, 1);
continue;
}
goto nomem;
}
copy_highpage(netpage, backpage);
put_page(backpage);
backpage = NULL;
fscache_mark_page_cached(op, netpage);
/* the netpage is unlocked and marked up to date here */
fscache_end_io(op, netpage, 0);
put_page(netpage);
netpage = NULL;
fscache_retrieval_complete(op, 1);
continue;
}
netpage = NULL;
_debug("out");
out:
/* tidy up */
if (newpage)
put_page(newpage);
if (netpage)
put_page(netpage);
if (backpage)
put_page(backpage);
if (monitor) {
fscache_put_retrieval(op);
kfree(monitor);
}
list_for_each_entry_safe(netpage, _n, list, lru) {
list_del(&netpage->lru);
put_page(netpage);
fscache_retrieval_complete(op, 1);
}
_leave(" = %d", ret);
return ret;
nomem:
_debug("nomem");
ret = -ENOMEM;
goto record_page_complete;
read_error:
_debug("read error %d", ret);
if (ret == -ENOMEM)
goto record_page_complete;
io_error:
cachefiles_io_error_obj(object, "Page read error on backing file");
ret = -ENOBUFS;
record_page_complete:
fscache_retrieval_complete(op, 1);
goto out;
}
/*
* read a list of pages from the cache or allocate blocks in which to store
* them
*/
int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op,
struct list_head *pages,
unsigned *nr_pages,
gfp_t gfp)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct list_head backpages;
struct pagevec pagevec;
struct inode *inode;
struct page *page, *_n;
unsigned shift, nrbackpages;
int ret, ret2, space;
object = container_of(op->op.object,
struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
_enter("{OBJ%x,%d},,%d,,",
object->fscache.debug_id, atomic_read(&op->op.usage),
*nr_pages);
if (!object->backer)
goto all_enobufs;
space = 1;
if (cachefiles_has_space(cache, 0, *nr_pages) < 0)
space = 0;
inode = d_backing_inode(object->backer);
ASSERT(S_ISREG(inode->i_mode));
/* calculate the shift required to use bmap */
shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits;
pagevec_init(&pagevec);
op->op.flags &= FSCACHE_OP_KEEP_FLAGS;
op->op.flags |= FSCACHE_OP_ASYNC;
op->op.processor = cachefiles_read_copier;
INIT_LIST_HEAD(&backpages);
nrbackpages = 0;
ret = space ? -ENODATA : -ENOBUFS;
list_for_each_entry_safe(page, _n, pages, lru) {
sector_t block;
/* we assume the absence or presence of the first block is a
* good enough indication for the page as a whole
* - TODO: don't use bmap() for this as it is _not_ actually
* good enough for this as it doesn't indicate errors, but
* it's all we've got for the moment
*/
block = page->index;
block <<= shift;
ret2 = bmap(inode, &block);
ASSERT(ret2 == 0);
_debug("%llx -> %llx",
(unsigned long long) (page->index << shift),
(unsigned long long) block);
if (block) {
/* we have data - add it to the list to give to the
* backing fs */
list_move(&page->lru, &backpages);
(*nr_pages)--;
nrbackpages++;
} else if (space && pagevec_add(&pagevec, page) == 0) {
fscache_mark_pages_cached(op, &pagevec);
fscache_retrieval_complete(op, 1);
ret = -ENODATA;
} else {
fscache_retrieval_complete(op, 1);
}
}
if (pagevec_count(&pagevec) > 0)
fscache_mark_pages_cached(op, &pagevec);
if (list_empty(pages))
ret = 0;
/* submit the apparently valid pages to the backing fs to be read from
* disk */
if (nrbackpages > 0) {
ret2 = cachefiles_read_backing_file(object, op, &backpages);
if (ret2 == -ENOMEM || ret2 == -EINTR)
ret = ret2;
}
_leave(" = %d [nr=%u%s]",
ret, *nr_pages, list_empty(pages) ? " empty" : "");
return ret;
all_enobufs:
fscache_retrieval_complete(op, *nr_pages);
return -ENOBUFS;
}
/*
* allocate a block in the cache in which to store a page
* - cache withdrawal is prevented by the caller
* - returns -EINTR if interrupted
* - returns -ENOMEM if ran out of memory
* - returns -ENOBUFS if no buffers can be made available
* - returns -ENOBUFS if page is beyond EOF
* - otherwise:
* - the metadata will be retained
* - 0 will be returned
*/
int cachefiles_allocate_page(struct fscache_retrieval *op,
struct page *page,
gfp_t gfp)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
int ret;
object = container_of(op->op.object,
struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
_enter("%p,{%lx},", object, page->index);
ret = cachefiles_has_space(cache, 0, 1);
if (ret == 0)
fscache_mark_page_cached(op, page);
else
ret = -ENOBUFS;
fscache_retrieval_complete(op, 1);
_leave(" = %d", ret);
return ret;
}
/*
* allocate blocks in the cache in which to store a set of pages
* - cache withdrawal is prevented by the caller
* - returns -EINTR if interrupted
* - returns -ENOMEM if ran out of memory
* - returns -ENOBUFS if some buffers couldn't be made available
* - returns -ENOBUFS if some pages are beyond EOF
* - otherwise:
* - -ENODATA will be returned
* - metadata will be retained for any page marked
*/
int cachefiles_allocate_pages(struct fscache_retrieval *op,
struct list_head *pages,
unsigned *nr_pages,
gfp_t gfp)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct pagevec pagevec;
struct page *page;
int ret;
object = container_of(op->op.object,
struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
_enter("%p,,,%d,", object, *nr_pages);
ret = cachefiles_has_space(cache, 0, *nr_pages);
if (ret == 0) {
pagevec_init(&pagevec);
list_for_each_entry(page, pages, lru) {
if (pagevec_add(&pagevec, page) == 0)
fscache_mark_pages_cached(op, &pagevec);
}
if (pagevec_count(&pagevec) > 0)
fscache_mark_pages_cached(op, &pagevec);
ret = -ENODATA;
} else {
ret = -ENOBUFS;
}
fscache_retrieval_complete(op, *nr_pages);
_leave(" = %d", ret);
return ret;
}
/*
* request a page be stored in the cache
* - cache withdrawal is prevented by the caller
* - this request may be ignored if there's no cache block available, in which
* case -ENOBUFS will be returned
* - if the op is in progress, 0 will be returned
*/
int cachefiles_write_page(struct fscache_storage *op, struct page *page)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct file *file;
struct path path;
loff_t pos, eof;
size_t len;
void *data;
int ret = -ENOBUFS;
ASSERT(op != NULL);
ASSERT(page != NULL);
object = container_of(op->op.object,
struct cachefiles_object, fscache);
_enter("%p,%p{%lx},,,", object, page, page->index);
if (!object->backer) {
_leave(" = -ENOBUFS");
return -ENOBUFS;
}
ASSERT(d_is_reg(object->backer));
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
pos = (loff_t)page->index << PAGE_SHIFT;
/* We mustn't write more data than we have, so we have to beware of a
* partial page at EOF.
*/
eof = object->fscache.store_limit_l;
if (pos >= eof)
goto error;
/* write the page to the backing filesystem and let it store it in its
* own time */
path.mnt = cache->mnt;
path.dentry = object->backer;
file = dentry_open(&path, O_RDWR | O_LARGEFILE, cache->cache_cred);
if (IS_ERR(file)) {
ret = PTR_ERR(file);
goto error_2;
}
len = PAGE_SIZE;
if (eof & ~PAGE_MASK) {
if (eof - pos < PAGE_SIZE) {
_debug("cut short %llx to %llx",
pos, eof);
len = eof - pos;
ASSERTCMP(pos + len, ==, eof);
}
}
data = kmap(page);
ret = kernel_write(file, data, len, &pos);
kunmap(page);
fput(file);
if (ret != len)
goto error_eio;
_leave(" = 0");
return 0;
error_eio:
ret = -EIO;
error_2:
if (ret == -EIO)
cachefiles_io_error_obj(object,
"Write page to backing file failed");
error:
_leave(" = -ENOBUFS [%d]", ret);
return -ENOBUFS;
}
/*
* detach a backing block from a page
* - cache withdrawal is prevented by the caller
*/
void cachefiles_uncache_page(struct fscache_object *_object, struct page *page)
__releases(&object->fscache.cookie->lock)
{
struct cachefiles_object *object;
object = container_of(_object, struct cachefiles_object, fscache);
_enter("%p,{%lu}", object, page->index);
spin_unlock(&object->fscache.cookie->lock);
}

2
fs/cachefiles/security.c
View File

@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* CacheFiles security management
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2007, 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/

139
fs/cachefiles/volume.c Normal file
View File

@ -0,0 +1,139 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Volume handling.
*
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/fs.h>
#include <linux/slab.h>
#include "internal.h"
#include <trace/events/fscache.h>
/*
* Allocate and set up a volume representation. We make sure all the fanout
* directories are created and pinned.
*/
void cachefiles_acquire_volume(struct fscache_volume *vcookie)
{
struct cachefiles_volume *volume;
struct cachefiles_cache *cache = vcookie->cache->cache_priv;
const struct cred *saved_cred;
struct dentry *vdentry, *fan;
size_t len;
char *name;
bool is_new = false;
int ret, n_accesses, i;
_enter("");
volume = kzalloc(sizeof(struct cachefiles_volume), GFP_KERNEL);
if (!volume)
return;
volume->vcookie = vcookie;
volume->cache = cache;
INIT_LIST_HEAD(&volume->cache_link);
cachefiles_begin_secure(cache, &saved_cred);
len = vcookie->key[0];
name = kmalloc(len + 3, GFP_NOFS);
if (!name)
goto error_vol;
name[0] = 'I';
memcpy(name + 1, vcookie->key + 1, len);
name[len + 1] = 0;
retry:
vdentry = cachefiles_get_directory(cache, cache->store, name, &is_new);
if (IS_ERR(vdentry))
goto error_name;
volume->dentry = vdentry;
if (is_new) {
if (!cachefiles_set_volume_xattr(volume))
goto error_dir;
} else {
ret = cachefiles_check_volume_xattr(volume);
if (ret < 0) {
if (ret != -ESTALE)
goto error_dir;
inode_lock_nested(d_inode(cache->store), I_MUTEX_PARENT);
cachefiles_bury_object(cache, NULL, cache->store, vdentry,
FSCACHE_VOLUME_IS_WEIRD);
cachefiles_put_directory(volume->dentry);
cond_resched();
goto retry;
}
}
for (i = 0; i < 256; i++) {
sprintf(name, "@%02x", i);
fan = cachefiles_get_directory(cache, vdentry, name, NULL);
if (IS_ERR(fan))
goto error_fan;
volume->fanout[i] = fan;
}
cachefiles_end_secure(cache, saved_cred);
vcookie->cache_priv = volume;
n_accesses = atomic_inc_return(&vcookie->n_accesses); /* Stop wakeups on dec-to-0 */
trace_fscache_access_volume(vcookie->debug_id, 0,
refcount_read(&vcookie->ref),
n_accesses, fscache_access_cache_pin);
spin_lock(&cache->object_list_lock);
list_add(&volume->cache_link, &volume->cache->volumes);
spin_unlock(&cache->object_list_lock);
kfree(name);
return;
error_fan:
for (i = 0; i < 256; i++)
cachefiles_put_directory(volume->fanout[i]);
error_dir:
cachefiles_put_directory(volume->dentry);
error_name:
kfree(name);
error_vol:
kfree(volume);
cachefiles_end_secure(cache, saved_cred);
}
/*
* Release a volume representation.
*/
static void __cachefiles_free_volume(struct cachefiles_volume *volume)
{
int i;
_enter("");
volume->vcookie->cache_priv = NULL;
for (i = 0; i < 256; i++)
cachefiles_put_directory(volume->fanout[i]);
cachefiles_put_directory(volume->dentry);
kfree(volume);
}
void cachefiles_free_volume(struct fscache_volume *vcookie)
{
struct cachefiles_volume *volume = vcookie->cache_priv;
if (volume) {
spin_lock(&volume->cache->object_list_lock);
list_del_init(&volume->cache_link);
spin_unlock(&volume->cache->object_list_lock);
__cachefiles_free_volume(volume);
}
}
void cachefiles_withdraw_volume(struct cachefiles_volume *volume)
{
fscache_withdraw_volume(volume->vcookie);
cachefiles_set_volume_xattr(volume);
__cachefiles_free_volume(volume);
}

449
fs/cachefiles/xattr.c
View File

@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* CacheFiles extended attribute management
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
@ -15,138 +15,70 @@
#include <linux/slab.h>
#include "internal.h"
#define CACHEFILES_COOKIE_TYPE_DATA 1
struct cachefiles_xattr {
__be64 object_size; /* Actual size of the object */
__be64 zero_point; /* Size after which server has no data not written by us */
__u8 type; /* Type of object */
__u8 content; /* Content presence (enum cachefiles_content) */
__u8 data[]; /* netfs coherency data */
} __packed;
static const char cachefiles_xattr_cache[] =
XATTR_USER_PREFIX "CacheFiles.cache";
/*
* check the type label on an object
* - done using xattrs
*/
int cachefiles_check_object_type(struct cachefiles_object *object)
{
struct dentry *dentry = object->dentry;
char type[3], xtype[3];
int ret;
ASSERT(dentry);
ASSERT(d_backing_inode(dentry));
if (!object->fscache.cookie)
strcpy(type, "C3");
else
snprintf(type, 3, "%02x", object->fscache.cookie->def->type);
_enter("%x{%s}", object->fscache.debug_id, type);
/* attempt to install a type label directly */
ret = vfs_setxattr(&init_user_ns, dentry, cachefiles_xattr_cache, type,
2, XATTR_CREATE);
if (ret == 0) {
_debug("SET"); /* we succeeded */
goto error;
}
if (ret != -EEXIST) {
pr_err("Can't set xattr on %pd [%lu] (err %d)\n",
dentry, d_backing_inode(dentry)->i_ino,
-ret);
goto error;
}
/* read the current type label */
ret = vfs_getxattr(&init_user_ns, dentry, cachefiles_xattr_cache, xtype,
3);
if (ret < 0) {
if (ret == -ERANGE)
goto bad_type_length;
pr_err("Can't read xattr on %pd [%lu] (err %d)\n",
dentry, d_backing_inode(dentry)->i_ino,
-ret);
goto error;
}
/* check the type is what we're expecting */
if (ret != 2)
goto bad_type_length;
if (xtype[0] != type[0] || xtype[1] != type[1])
goto bad_type;
ret = 0;
error:
_leave(" = %d", ret);
return ret;
bad_type_length:
pr_err("Cache object %lu type xattr length incorrect\n",
d_backing_inode(dentry)->i_ino);
ret = -EIO;
goto error;
bad_type:
xtype[2] = 0;
pr_err("Cache object %pd [%lu] type %s not %s\n",
dentry, d_backing_inode(dentry)->i_ino,
xtype, type);
ret = -EIO;
goto error;
}
/*
* set the state xattr on a cache file
*/
int cachefiles_set_object_xattr(struct cachefiles_object *object,
struct cachefiles_xattr *auxdata)
int cachefiles_set_object_xattr(struct cachefiles_object *object)
{
struct dentry *dentry = object->dentry;
struct cachefiles_xattr *buf;
struct dentry *dentry;
struct file *file = object->file;
unsigned int len = object->cookie->aux_len;
int ret;
ASSERT(dentry);
_enter("%p,#%d", object, auxdata->len);
/* attempt to install the cache metadata directly */
_debug("SET #%u", auxdata->len);
clear_bit(FSCACHE_COOKIE_AUX_UPDATED, &object->fscache.cookie->flags);
ret = vfs_setxattr(&init_user_ns, dentry, cachefiles_xattr_cache,
&auxdata->type, auxdata->len, XATTR_CREATE);
if (ret < 0 && ret != -ENOMEM)
cachefiles_io_error_obj(
object,
"Failed to set xattr with error %d", ret);
_leave(" = %d", ret);
return ret;
}
/*
* update the state xattr on a cache file
*/
int cachefiles_update_object_xattr(struct cachefiles_object *object,
struct cachefiles_xattr *auxdata)
{
struct dentry *dentry = object->dentry;
int ret;
if (!dentry)
if (!file)
return -ESTALE;
dentry = file->f_path.dentry;
_enter("%x,#%d", object->fscache.debug_id, auxdata->len);
_enter("%x,#%d", object->debug_id, len);
/* attempt to install the cache metadata directly */
_debug("SET #%u", auxdata->len);
buf = kmalloc(sizeof(struct cachefiles_xattr) + len, GFP_KERNEL);
if (!buf)
return -ENOMEM;
clear_bit(FSCACHE_COOKIE_AUX_UPDATED, &object->fscache.cookie->flags);
ret = vfs_setxattr(&init_user_ns, dentry, cachefiles_xattr_cache,
&auxdata->type, auxdata->len, XATTR_REPLACE);
if (ret < 0 && ret != -ENOMEM)
cachefiles_io_error_obj(
object,
"Failed to update xattr with error %d", ret);
buf->object_size = cpu_to_be64(object->cookie->object_size);
buf->zero_point = 0;
buf->type = CACHEFILES_COOKIE_TYPE_DATA;
buf->content = object->content_info;
if (test_bit(FSCACHE_COOKIE_LOCAL_WRITE, &object->cookie->flags))
buf->content = CACHEFILES_CONTENT_DIRTY;
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->cookie), len);
ret = cachefiles_inject_write_error();
if (ret == 0)
ret = vfs_setxattr(&init_user_ns, dentry, cachefiles_xattr_cache,
buf, sizeof(struct cachefiles_xattr) + len, 0);
if (ret < 0) {
trace_cachefiles_vfs_error(object, file_inode(file), ret,
cachefiles_trace_setxattr_error);
trace_cachefiles_coherency(object, file_inode(file)->i_ino,
buf->content,
cachefiles_coherency_set_fail);
if (ret != -ENOMEM)
cachefiles_io_error_obj(
object,
"Failed to set xattr with error %d", ret);
} else {
trace_cachefiles_coherency(object, file_inode(file)->i_ino,
buf->content,
cachefiles_coherency_set_ok);
}
kfree(buf);
_leave(" = %d", ret);
return ret;
}
@ -154,162 +86,69 @@ int cachefiles_update_object_xattr(struct cachefiles_object *object,
/*
* check the consistency between the backing cache and the FS-Cache cookie
*/
int cachefiles_check_auxdata(struct cachefiles_object *object)
int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file)
{
struct cachefiles_xattr *auxbuf;
enum fscache_checkaux validity;
struct dentry *dentry = object->dentry;
struct cachefiles_xattr *buf;
struct dentry *dentry = file->f_path.dentry;
unsigned int len = object->cookie->aux_len, tlen;
const void *p = fscache_get_aux(object->cookie);
enum cachefiles_coherency_trace why;
ssize_t xlen;
int ret;
int ret = -ESTALE;
ASSERT(dentry);
ASSERT(d_backing_inode(dentry));
ASSERT(object->fscache.cookie->def->check_aux);
auxbuf = kmalloc(sizeof(struct cachefiles_xattr) + 512, GFP_KERNEL);
if (!auxbuf)
tlen = sizeof(struct cachefiles_xattr) + len;
buf = kmalloc(tlen, GFP_KERNEL);
if (!buf)
return -ENOMEM;
xlen = vfs_getxattr(&init_user_ns, dentry, cachefiles_xattr_cache,
&auxbuf->type, 512 + 1);
ret = -ESTALE;
if (xlen < 1 ||
auxbuf->type != object->fscache.cookie->def->type)
goto error;
xlen = cachefiles_inject_read_error();
if (xlen == 0)
xlen = vfs_getxattr(&init_user_ns, dentry, cachefiles_xattr_cache, buf, tlen);
if (xlen != tlen) {
if (xlen < 0)
trace_cachefiles_vfs_error(object, file_inode(file), xlen,
cachefiles_trace_getxattr_error);
if (xlen == -EIO)
cachefiles_io_error_obj(
object,
"Failed to read aux with error %zd", xlen);
why = cachefiles_coherency_check_xattr;
} else if (buf->type != CACHEFILES_COOKIE_TYPE_DATA) {
why = cachefiles_coherency_check_type;
} else if (memcmp(buf->data, p, len) != 0) {
why = cachefiles_coherency_check_aux;
} else if (be64_to_cpu(buf->object_size) != object->cookie->object_size) {
why = cachefiles_coherency_check_objsize;
} else if (buf->content == CACHEFILES_CONTENT_DIRTY) {
// TODO: Begin conflict resolution
pr_warn("Dirty object in cache\n");
why = cachefiles_coherency_check_dirty;
} else {
why = cachefiles_coherency_check_ok;
ret = 0;
}
xlen--;
validity = fscache_check_aux(&object->fscache, &auxbuf->data, xlen,
i_size_read(d_backing_inode(dentry)));
if (validity != FSCACHE_CHECKAUX_OKAY)
goto error;
ret = 0;
error:
kfree(auxbuf);
trace_cachefiles_coherency(object, file_inode(file)->i_ino,
buf->content, why);
kfree(buf);
return ret;
}
/*
* check the state xattr on a cache file
* - return -ESTALE if the object should be deleted
*/
int cachefiles_check_object_xattr(struct cachefiles_object *object,
struct cachefiles_xattr *auxdata)
{
struct cachefiles_xattr *auxbuf;
struct dentry *dentry = object->dentry;
int ret;
_enter("%p,#%d", object, auxdata->len);
ASSERT(dentry);
ASSERT(d_backing_inode(dentry));
auxbuf = kmalloc(sizeof(struct cachefiles_xattr) + 512, cachefiles_gfp);
if (!auxbuf) {
_leave(" = -ENOMEM");
return -ENOMEM;
}
/* read the current type label */
ret = vfs_getxattr(&init_user_ns, dentry, cachefiles_xattr_cache,
&auxbuf->type, 512 + 1);
if (ret < 0) {
if (ret == -ENODATA)
goto stale; /* no attribute - power went off
* mid-cull? */
if (ret == -ERANGE)
goto bad_type_length;
cachefiles_io_error_obj(object,
"Can't read xattr on %lu (err %d)",
d_backing_inode(dentry)->i_ino, -ret);
goto error;
}
/* check the on-disk object */
if (ret < 1)
goto bad_type_length;
if (auxbuf->type != auxdata->type)
goto stale;
auxbuf->len = ret;
/* consult the netfs */
if (object->fscache.cookie->def->check_aux) {
enum fscache_checkaux result;
unsigned int dlen;
dlen = auxbuf->len - 1;
_debug("checkaux %s #%u",
object->fscache.cookie->def->name, dlen);
result = fscache_check_aux(&object->fscache,
&auxbuf->data, dlen,
i_size_read(d_backing_inode(dentry)));
switch (result) {
/* entry okay as is */
case FSCACHE_CHECKAUX_OKAY:
goto okay;
/* entry requires update */
case FSCACHE_CHECKAUX_NEEDS_UPDATE:
break;
/* entry requires deletion */
case FSCACHE_CHECKAUX_OBSOLETE:
goto stale;
default:
BUG();
}
/* update the current label */
ret = vfs_setxattr(&init_user_ns, dentry,
cachefiles_xattr_cache, &auxdata->type,
auxdata->len, XATTR_REPLACE);
if (ret < 0) {
cachefiles_io_error_obj(object,
"Can't update xattr on %lu"
" (error %d)",
d_backing_inode(dentry)->i_ino, -ret);
goto error;
}
}
okay:
ret = 0;
error:
kfree(auxbuf);
_leave(" = %d", ret);
return ret;
bad_type_length:
pr_err("Cache object %lu xattr length incorrect\n",
d_backing_inode(dentry)->i_ino);
ret = -EIO;
goto error;
stale:
ret = -ESTALE;
goto error;
}
/*
* remove the object's xattr to mark it stale
*/
int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
struct cachefiles_object *object,
struct dentry *dentry)
{
int ret;
ret = vfs_removexattr(&init_user_ns, dentry, cachefiles_xattr_cache);
ret = cachefiles_inject_remove_error();
if (ret == 0)
ret = vfs_removexattr(&init_user_ns, dentry, cachefiles_xattr_cache);
if (ret < 0) {
trace_cachefiles_vfs_error(object, d_inode(dentry), ret,
cachefiles_trace_remxattr_error);
if (ret == -ENOENT || ret == -ENODATA)
ret = 0;
else if (ret != -ENOMEM)
@ -322,3 +161,99 @@ int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
_leave(" = %d", ret);
return ret;
}
/*
* Stick a marker on the cache object to indicate that it's dirty.
*/
void cachefiles_prepare_to_write(struct fscache_cookie *cookie)
{
const struct cred *saved_cred;
struct cachefiles_object *object = cookie->cache_priv;
struct cachefiles_cache *cache = object->volume->cache;
_enter("c=%08x", object->cookie->debug_id);
if (!test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags)) {
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_set_object_xattr(object);
cachefiles_end_secure(cache, saved_cred);
}
}
/*
* Set the state xattr on a volume directory.
*/
bool cachefiles_set_volume_xattr(struct cachefiles_volume *volume)
{
unsigned int len = volume->vcookie->coherency_len;
const void *p = volume->vcookie->coherency;
struct dentry *dentry = volume->dentry;
int ret;
_enter("%x,#%d", volume->vcookie->debug_id, len);
ret = cachefiles_inject_write_error();
if (ret == 0)
ret = vfs_setxattr(&init_user_ns, dentry, cachefiles_xattr_cache,
p, len, 0);
if (ret < 0) {
trace_cachefiles_vfs_error(NULL, d_inode(dentry), ret,
cachefiles_trace_setxattr_error);
trace_cachefiles_vol_coherency(volume, d_inode(dentry)->i_ino,
cachefiles_coherency_vol_set_fail);
if (ret != -ENOMEM)
cachefiles_io_error(
volume->cache, "Failed to set xattr with error %d", ret);
} else {
trace_cachefiles_vol_coherency(volume, d_inode(dentry)->i_ino,
cachefiles_coherency_vol_set_ok);
}
_leave(" = %d", ret);
return ret == 0;
}
/*
* Check the consistency between the backing cache and the volume cookie.
*/
int cachefiles_check_volume_xattr(struct cachefiles_volume *volume)
{
struct cachefiles_xattr *buf;
struct dentry *dentry = volume->dentry;
unsigned int len = volume->vcookie->coherency_len;
const void *p = volume->vcookie->coherency;
enum cachefiles_coherency_trace why;
ssize_t xlen;
int ret = -ESTALE;
_enter("");
buf = kmalloc(len, GFP_KERNEL);
if (!buf)
return -ENOMEM;
xlen = cachefiles_inject_read_error();
if (xlen == 0)
xlen = vfs_getxattr(&init_user_ns, dentry, cachefiles_xattr_cache, buf, len);
if (xlen != len) {
if (xlen < 0) {
trace_cachefiles_vfs_error(NULL, d_inode(dentry), xlen,
cachefiles_trace_getxattr_error);
if (xlen == -EIO)
cachefiles_io_error(
volume->cache,
"Failed to read xattr with error %zd", xlen);
}
why = cachefiles_coherency_vol_check_xattr;
} else if (memcmp(buf->data, p, len) != 0) {
why = cachefiles_coherency_vol_check_cmp;
} else {
why = cachefiles_coherency_vol_check_ok;
ret = 0;
}
trace_cachefiles_vol_coherency(volume, d_inode(dentry)->i_ino, why);
kfree(buf);
_leave(" = %d", ret);
return ret;
}

102
fs/ceph/addr.c
View File

@ -4,8 +4,8 @@
#include <linux/backing-dev.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/pagemap.h>
#include <linux/writeback.h> /* generic_writepages */
#include <linux/slab.h>
#include <linux/pagevec.h>
#include <linux/task_io_accounting_ops.h>
@ -126,7 +126,7 @@ static int ceph_set_page_dirty(struct page *page)
BUG_ON(PagePrivate(page));
attach_page_private(page, snapc);
return __set_page_dirty_nobuffers(page);
return ceph_fscache_set_page_dirty(page);
}
/*
@ -141,8 +141,6 @@ static void ceph_invalidatepage(struct page *page, unsigned int offset,
struct ceph_inode_info *ci;
struct ceph_snap_context *snapc;
wait_on_page_fscache(page);
inode = page->mapping->host;
ci = ceph_inode(inode);
@ -153,28 +151,36 @@ static void ceph_invalidatepage(struct page *page, unsigned int offset,
}
WARN_ON(!PageLocked(page));
if (!PagePrivate(page))
return;
if (PagePrivate(page)) {
dout("%p invalidatepage %p idx %lu full dirty page\n",
inode, page, page->index);
dout("%p invalidatepage %p idx %lu full dirty page\n",
inode, page, page->index);
snapc = detach_page_private(page);
ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
ceph_put_snap_context(snapc);
}
snapc = detach_page_private(page);
ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
ceph_put_snap_context(snapc);
wait_on_page_fscache(page);
}
static int ceph_releasepage(struct page *page, gfp_t gfp)
{
dout("%p releasepage %p idx %lu (%sdirty)\n", page->mapping->host,
page, page->index, PageDirty(page) ? "" : "not ");
struct inode *inode = page->mapping->host;
dout("%llx:%llx releasepage %p idx %lu (%sdirty)\n",
ceph_vinop(inode), page,
page->index, PageDirty(page) ? "" : "not ");
if (PagePrivate(page))
return 0;
if (PageFsCache(page)) {
if (!(gfp & __GFP_DIRECT_RECLAIM) || !(gfp & __GFP_FS))
if (current_is_kswapd() || !(gfp & __GFP_FS))
return 0;
wait_on_page_fscache(page);
}
return !PagePrivate(page);
ceph_fscache_note_page_release(inode);
return 1;
}
static void ceph_netfs_expand_readahead(struct netfs_read_request *rreq)
@ -378,6 +384,38 @@ static void ceph_readahead(struct readahead_control *ractl)
netfs_readahead(ractl, &ceph_netfs_read_ops, (void *)(uintptr_t)got);
}
#ifdef CONFIG_CEPH_FSCACHE
static void ceph_set_page_fscache(struct page *page)
{
set_page_fscache(page);
}
static void ceph_fscache_write_terminated(void *priv, ssize_t error, bool was_async)
{
struct inode *inode = priv;
if (IS_ERR_VALUE(error) && error != -ENOBUFS)
ceph_fscache_invalidate(inode, false);
}
static void ceph_fscache_write_to_cache(struct inode *inode, u64 off, u64 len, bool caching)
{
struct ceph_inode_info *ci = ceph_inode(inode);
struct fscache_cookie *cookie = ceph_fscache_cookie(ci);
fscache_write_to_cache(cookie, inode->i_mapping, off, len, i_size_read(inode),
ceph_fscache_write_terminated, inode, caching);
}
#else
static inline void ceph_set_page_fscache(struct page *page)
{
}
static inline void ceph_fscache_write_to_cache(struct inode *inode, u64 off, u64 len, bool caching)
{
}
#endif /* CONFIG_CEPH_FSCACHE */
struct ceph_writeback_ctl
{
loff_t i_size;
@ -493,6 +531,7 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
struct ceph_writeback_ctl ceph_wbc;
struct ceph_osd_client *osdc = &fsc->client->osdc;
struct ceph_osd_request *req;
bool caching = ceph_is_cache_enabled(inode);
dout("writepage %p idx %lu\n", page, page->index);
@ -531,16 +570,17 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb))
set_bdi_congested(inode_to_bdi(inode), BLK_RW_ASYNC);
set_page_writeback(page);
req = ceph_osdc_new_request(osdc, &ci->i_layout, ceph_vino(inode), page_off, &len, 0, 1,
CEPH_OSD_OP_WRITE, CEPH_OSD_FLAG_WRITE, snapc,
ceph_wbc.truncate_seq, ceph_wbc.truncate_size,
true);
if (IS_ERR(req)) {
redirty_page_for_writepage(wbc, page);
end_page_writeback(page);
if (IS_ERR(req))
return PTR_ERR(req);
}
set_page_writeback(page);
if (caching)
ceph_set_page_fscache(page);
ceph_fscache_write_to_cache(inode, page_off, len, caching);
/* it may be a short write due to an object boundary */
WARN_ON_ONCE(len > thp_size(page));
@ -599,6 +639,9 @@ static int ceph_writepage(struct page *page, struct writeback_control *wbc)
struct inode *inode = page->mapping->host;
BUG_ON(!inode);
ihold(inode);
wait_on_page_fscache(page);
err = writepage_nounlock(page, wbc);
if (err == -ERESTARTSYS) {
/* direct memory reclaimer was killed by SIGKILL. return 0
@ -720,6 +763,7 @@ static int ceph_writepages_start(struct address_space *mapping,
struct ceph_writeback_ctl ceph_wbc;
bool should_loop, range_whole = false;
bool done = false;
bool caching = ceph_is_cache_enabled(inode);
dout("writepages_start %p (mode=%s)\n", inode,
wbc->sync_mode == WB_SYNC_NONE ? "NONE" :
@ -843,7 +887,7 @@ static int ceph_writepages_start(struct address_space *mapping,
unlock_page(page);
break;
}
if (PageWriteback(page)) {
if (PageWriteback(page) || PageFsCache(page)) {
if (wbc->sync_mode == WB_SYNC_NONE) {
dout("%p under writeback\n", page);
unlock_page(page);
@ -851,6 +895,7 @@ static int ceph_writepages_start(struct address_space *mapping,
}
dout("waiting on writeback %p\n", page);
wait_on_page_writeback(page);
wait_on_page_fscache(page);
}
if (!clear_page_dirty_for_io(page)) {
@ -983,9 +1028,19 @@ static int ceph_writepages_start(struct address_space *mapping,
op_idx = 0;
for (i = 0; i < locked_pages; i++) {
u64 cur_offset = page_offset(pages[i]);
/*
* Discontinuity in page range? Ceph can handle that by just passing
* multiple extents in the write op.
*/
if (offset + len != cur_offset) {
/* If it's full, stop here */
if (op_idx + 1 == req->r_num_ops)
break;
/* Kick off an fscache write with what we have so far. */
ceph_fscache_write_to_cache(inode, offset, len, caching);
/* Start a new extent */
osd_req_op_extent_dup_last(req, op_idx,
cur_offset - offset);
dout("writepages got pages at %llu~%llu\n",
@ -996,14 +1051,17 @@ static int ceph_writepages_start(struct address_space *mapping,
osd_req_op_extent_update(req, op_idx, len);
len = 0;
offset = cur_offset;
offset = cur_offset;
data_pages = pages + i;
op_idx++;
}
set_page_writeback(pages[i]);
if (caching)
ceph_set_page_fscache(pages[i]);
len += thp_size(page);
}
ceph_fscache_write_to_cache(inode, offset, len, caching);
if (ceph_wbc.size_stable) {
len = min(len, ceph_wbc.i_size - offset);

250
fs/ceph/cache.c
View File

@ -12,36 +12,73 @@
#include "super.h"
#include "cache.h"
struct fscache_netfs ceph_cache_netfs = {
.name = "ceph",
.version = 0,
};
static DEFINE_MUTEX(ceph_fscache_lock);
static LIST_HEAD(ceph_fscache_list);
struct ceph_fscache_entry {
struct list_head list;
struct fscache_cookie *fscache;
size_t uniq_len;
/* The following members must be last */
struct ceph_fsid fsid;
char uniquifier[];
};
static const struct fscache_cookie_def ceph_fscache_fsid_object_def = {
.name = "CEPH.fsid",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
int __init ceph_fscache_register(void)
void ceph_fscache_register_inode_cookie(struct inode *inode)
{
return fscache_register_netfs(&ceph_cache_netfs);
struct ceph_inode_info *ci = ceph_inode(inode);
struct ceph_fs_client *fsc = ceph_inode_to_client(inode);
/* No caching for filesystem? */
if (!fsc->fscache)
return;
/* Regular files only */
if (!S_ISREG(inode->i_mode))
return;
/* Only new inodes! */
if (!(inode->i_state & I_NEW))
return;
WARN_ON_ONCE(ci->fscache);
ci->fscache = fscache_acquire_cookie(fsc->fscache, 0,
&ci->i_vino, sizeof(ci->i_vino),
&ci->i_version, sizeof(ci->i_version),
i_size_read(inode));
}
void ceph_fscache_unregister(void)
void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info* ci)
{
fscache_unregister_netfs(&ceph_cache_netfs);
struct fscache_cookie *cookie = ci->fscache;
fscache_relinquish_cookie(cookie, false);
}
void ceph_fscache_use_cookie(struct inode *inode, bool will_modify)
{
struct ceph_inode_info *ci = ceph_inode(inode);
fscache_use_cookie(ci->fscache, will_modify);
}
void ceph_fscache_unuse_cookie(struct inode *inode, bool update)
{
struct ceph_inode_info *ci = ceph_inode(inode);
if (update) {
loff_t i_size = i_size_read(inode);
fscache_unuse_cookie(ci->fscache, &ci->i_version, &i_size);
} else {
fscache_unuse_cookie(ci->fscache, NULL, NULL);
}
}
void ceph_fscache_update(struct inode *inode)
{
struct ceph_inode_info *ci = ceph_inode(inode);
loff_t i_size = i_size_read(inode);
fscache_update_cookie(ci->fscache, &ci->i_version, &i_size);
}
void ceph_fscache_invalidate(struct inode *inode, bool dio_write)
{
struct ceph_inode_info *ci = ceph_inode(inode);
fscache_invalidate(ceph_inode(inode)->fscache,
&ci->i_version, i_size_read(inode),
dio_write ? FSCACHE_INVAL_DIO_WRITE : 0);
}
int ceph_fscache_register_fs(struct ceph_fs_client* fsc, struct fs_context *fc)
@ -49,162 +86,25 @@ int ceph_fscache_register_fs(struct ceph_fs_client* fsc, struct fs_context *fc)
const struct ceph_fsid *fsid = &fsc->client->fsid;
const char *fscache_uniq = fsc->mount_options->fscache_uniq;
size_t uniq_len = fscache_uniq ? strlen(fscache_uniq) : 0;
struct ceph_fscache_entry *ent;
char *name;
int err = 0;
mutex_lock(&ceph_fscache_lock);
list_for_each_entry(ent, &ceph_fscache_list, list) {
if (memcmp(&ent->fsid, fsid, sizeof(*fsid)))
continue;
if (ent->uniq_len != uniq_len)
continue;
if (uniq_len && memcmp(ent->uniquifier, fscache_uniq, uniq_len))
continue;
name = kasprintf(GFP_KERNEL, "ceph,%pU%s%s", fsid, uniq_len ? "," : "",
uniq_len ? fscache_uniq : "");
if (!name)
return -ENOMEM;
errorfc(fc, "fscache cookie already registered for fsid %pU, use fsc=<uniquifier> option",
fsid);
err = -EBUSY;
goto out_unlock;
fsc->fscache = fscache_acquire_volume(name, NULL, NULL, 0);
if (IS_ERR_OR_NULL(fsc->fscache)) {
errorfc(fc, "Unable to register fscache cookie for %s", name);
err = fsc->fscache ? PTR_ERR(fsc->fscache) : -EOPNOTSUPP;
fsc->fscache = NULL;
}
ent = kzalloc(sizeof(*ent) + uniq_len, GFP_KERNEL);
if (!ent) {
err = -ENOMEM;
goto out_unlock;
}
memcpy(&ent->fsid, fsid, sizeof(*fsid));
if (uniq_len > 0) {
memcpy(&ent->uniquifier, fscache_uniq, uniq_len);
ent->uniq_len = uniq_len;
}
fsc->fscache = fscache_acquire_cookie(ceph_cache_netfs.primary_index,
&ceph_fscache_fsid_object_def,
&ent->fsid, sizeof(ent->fsid) + uniq_len,
NULL, 0,
fsc, 0, true);
if (fsc->fscache) {
ent->fscache = fsc->fscache;
list_add_tail(&ent->list, &ceph_fscache_list);
} else {
kfree(ent);
errorfc(fc, "unable to register fscache cookie for fsid %pU",
fsid);
/* all other fs ignore this error */
}
out_unlock:
mutex_unlock(&ceph_fscache_lock);
kfree(name);
return err;
}
static enum fscache_checkaux ceph_fscache_inode_check_aux(
void *cookie_netfs_data, const void *data, uint16_t dlen,
loff_t object_size)
{
struct ceph_inode_info* ci = cookie_netfs_data;
struct inode* inode = &ci->vfs_inode;
if (dlen != sizeof(ci->i_version) ||
i_size_read(inode) != object_size)
return FSCACHE_CHECKAUX_OBSOLETE;
if (*(u64 *)data != ci->i_version)
return FSCACHE_CHECKAUX_OBSOLETE;
dout("ceph inode 0x%p cached okay\n", ci);
return FSCACHE_CHECKAUX_OKAY;
}
static const struct fscache_cookie_def ceph_fscache_inode_object_def = {
.name = "CEPH.inode",
.type = FSCACHE_COOKIE_TYPE_DATAFILE,
.check_aux = ceph_fscache_inode_check_aux,
};
void ceph_fscache_register_inode_cookie(struct inode *inode)
{
struct ceph_inode_info *ci = ceph_inode(inode);
struct ceph_fs_client *fsc = ceph_inode_to_client(inode);
/* No caching for filesystem */
if (!fsc->fscache)
return;
/* Only cache for regular files that are read only */
if (!S_ISREG(inode->i_mode))
return;
inode_lock_nested(inode, I_MUTEX_CHILD);
if (!ci->fscache) {
ci->fscache = fscache_acquire_cookie(fsc->fscache,
&ceph_fscache_inode_object_def,
&ci->i_vino, sizeof(ci->i_vino),
&ci->i_version, sizeof(ci->i_version),
ci, i_size_read(inode), false);
}
inode_unlock(inode);
}
void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info* ci)
{
struct fscache_cookie* cookie;
if ((cookie = ci->fscache) == NULL)
return;
ci->fscache = NULL;
fscache_relinquish_cookie(cookie, &ci->i_vino, false);
}
static bool ceph_fscache_can_enable(void *data)
{
struct inode *inode = data;
return !inode_is_open_for_write(inode);
}
void ceph_fscache_file_set_cookie(struct inode *inode, struct file *filp)
{
struct ceph_inode_info *ci = ceph_inode(inode);
if (!fscache_cookie_valid(ci->fscache))
return;
if (inode_is_open_for_write(inode)) {
dout("fscache_file_set_cookie %p %p disabling cache\n",
inode, filp);
fscache_disable_cookie(ci->fscache, &ci->i_vino, false);
} else {
fscache_enable_cookie(ci->fscache, &ci->i_vino, i_size_read(inode),
ceph_fscache_can_enable, inode);
if (fscache_cookie_enabled(ci->fscache)) {
dout("fscache_file_set_cookie %p %p enabling cache\n",
inode, filp);
}
}
}
void ceph_fscache_unregister_fs(struct ceph_fs_client* fsc)
{
if (fscache_cookie_valid(fsc->fscache)) {
struct ceph_fscache_entry *ent;
bool found = false;
mutex_lock(&ceph_fscache_lock);
list_for_each_entry(ent, &ceph_fscache_list, list) {
if (ent->fscache == fsc->fscache) {
list_del(&ent->list);
kfree(ent);
found = true;
break;
}
}
WARN_ON_ONCE(!found);
mutex_unlock(&ceph_fscache_lock);
__fscache_relinquish_cookie(fsc->fscache, NULL, false);
}
fsc->fscache = NULL;
fscache_relinquish_volume(fsc->fscache, NULL, false);
}

99
fs/ceph/cache.h
View File

@ -12,19 +12,19 @@
#include <linux/netfs.h>
#ifdef CONFIG_CEPH_FSCACHE
extern struct fscache_netfs ceph_cache_netfs;
int ceph_fscache_register(void);
void ceph_fscache_unregister(void);
#include <linux/fscache.h>
int ceph_fscache_register_fs(struct ceph_fs_client* fsc, struct fs_context *fc);
void ceph_fscache_unregister_fs(struct ceph_fs_client* fsc);
void ceph_fscache_register_inode_cookie(struct inode *inode);
void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info* ci);
void ceph_fscache_file_set_cookie(struct inode *inode, struct file *filp);
void ceph_fscache_revalidate_cookie(struct ceph_inode_info *ci);
void ceph_fscache_use_cookie(struct inode *inode, bool will_modify);
void ceph_fscache_unuse_cookie(struct inode *inode, bool update);
void ceph_fscache_update(struct inode *inode);
void ceph_fscache_invalidate(struct inode *inode, bool dio_write);
static inline void ceph_fscache_inode_init(struct ceph_inode_info *ci)
{
@ -36,37 +36,51 @@ static inline struct fscache_cookie *ceph_fscache_cookie(struct ceph_inode_info
return ci->fscache;
}
static inline void ceph_fscache_invalidate(struct inode *inode)
static inline void ceph_fscache_resize(struct inode *inode, loff_t to)
{
fscache_invalidate(ceph_inode(inode)->fscache);
struct ceph_inode_info *ci = ceph_inode(inode);
struct fscache_cookie *cookie = ceph_fscache_cookie(ci);
if (cookie) {
ceph_fscache_use_cookie(inode, true);
fscache_resize_cookie(cookie, to);
ceph_fscache_unuse_cookie(inode, true);
}
}
static inline bool ceph_is_cache_enabled(struct inode *inode)
static inline void ceph_fscache_unpin_writeback(struct inode *inode,
struct writeback_control *wbc)
{
struct fscache_cookie *cookie = ceph_fscache_cookie(ceph_inode(inode));
fscache_unpin_writeback(wbc, ceph_fscache_cookie(ceph_inode(inode)));
}
if (!cookie)
return false;
return fscache_cookie_enabled(cookie);
static inline int ceph_fscache_set_page_dirty(struct page *page)
{
struct inode *inode = page->mapping->host;
struct ceph_inode_info *ci = ceph_inode(inode);
return fscache_set_page_dirty(page, ceph_fscache_cookie(ci));
}
static inline int ceph_begin_cache_operation(struct netfs_read_request *rreq)
{
struct fscache_cookie *cookie = ceph_fscache_cookie(ceph_inode(rreq->inode));
return fscache_begin_read_operation(rreq, cookie);
return fscache_begin_read_operation(&rreq->cache_resources, cookie);
}
#else
static inline int ceph_fscache_register(void)
static inline bool ceph_is_cache_enabled(struct inode *inode)
{
return 0;
return fscache_cookie_enabled(ceph_fscache_cookie(ceph_inode(inode)));
}
static inline void ceph_fscache_unregister(void)
static inline void ceph_fscache_note_page_release(struct inode *inode)
{
}
struct ceph_inode_info *ci = ceph_inode(inode);
fscache_note_page_release(ceph_fscache_cookie(ci));
}
#else /* CONFIG_CEPH_FSCACHE */
static inline int ceph_fscache_register_fs(struct ceph_fs_client* fsc,
struct fs_context *fc)
{
@ -81,11 +95,6 @@ static inline void ceph_fscache_inode_init(struct ceph_inode_info *ci)
{
}
static inline struct fscache_cookie *ceph_fscache_cookie(struct ceph_inode_info *ci)
{
return NULL;
}
static inline void ceph_fscache_register_inode_cookie(struct inode *inode)
{
}
@ -94,15 +103,41 @@ static inline void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info*
{
}
static inline void ceph_fscache_file_set_cookie(struct inode *inode,
struct file *filp)
static inline void ceph_fscache_use_cookie(struct inode *inode, bool will_modify)
{
}
static inline void ceph_fscache_invalidate(struct inode *inode)
static inline void ceph_fscache_unuse_cookie(struct inode *inode, bool update)
{
}
static inline void ceph_fscache_update(struct inode *inode)
{
}
static inline void ceph_fscache_invalidate(struct inode *inode, bool dio_write)
{
}
static inline struct fscache_cookie *ceph_fscache_cookie(struct ceph_inode_info *ci)
{
return NULL;
}
static inline void ceph_fscache_resize(struct inode *inode, loff_t to)
{
}
static inline void ceph_fscache_unpin_writeback(struct inode *inode,
struct writeback_control *wbc)
{
}
static inline int ceph_fscache_set_page_dirty(struct page *page)
{
return __set_page_dirty_nobuffers(page);
}
static inline bool ceph_is_cache_enabled(struct inode *inode)
{
return false;
@ -112,6 +147,10 @@ static inline int ceph_begin_cache_operation(struct netfs_read_request *rreq)
{
return -ENOBUFS;
}
#endif
#endif /* _CEPH_CACHE_H */
static inline void ceph_fscache_note_page_release(struct inode *inode)
{
}
#endif /* CONFIG_CEPH_FSCACHE */
#endif

3
fs/ceph/caps.c
View File

@ -1856,7 +1856,7 @@ static int try_nonblocking_invalidate(struct inode *inode)
u32 invalidating_gen = ci->i_rdcache_gen;
spin_unlock(&ci->i_ceph_lock);
ceph_fscache_invalidate(inode);
ceph_fscache_invalidate(inode, false);
invalidate_mapping_pages(&inode->i_data, 0, -1);
spin_lock(&ci->i_ceph_lock);
@ -2388,6 +2388,7 @@ int ceph_write_inode(struct inode *inode, struct writeback_control *wbc)
int wait = (wbc->sync_mode == WB_SYNC_ALL && !wbc->for_sync);
dout("write_inode %p wait=%d\n", inode, wait);
ceph_fscache_unpin_writeback(inode, wbc);
if (wait) {
dirty = try_flush_caps(inode, &flush_tid);
if (dirty)

13
fs/ceph/file.c
View File

@ -248,8 +248,7 @@ static int ceph_init_file(struct inode *inode, struct file *file, int fmode)
switch (inode->i_mode & S_IFMT) {
case S_IFREG:
ceph_fscache_register_inode_cookie(inode);
ceph_fscache_file_set_cookie(inode, file);
ceph_fscache_use_cookie(inode, file->f_mode & FMODE_WRITE);
fallthrough;
case S_IFDIR:
ret = ceph_init_file_info(inode, file, fmode,
@ -822,6 +821,7 @@ int ceph_release(struct inode *inode, struct file *file)
dout("release inode %p regular file %p\n", inode, file);
WARN_ON(!list_empty(&fi->rw_contexts));
ceph_fscache_unuse_cookie(inode, file->f_mode & FMODE_WRITE);
ceph_put_fmode(ci, fi->fmode, 1);
kmem_cache_free(ceph_file_cachep, fi);
@ -1218,7 +1218,11 @@ ceph_direct_read_write(struct kiocb *iocb, struct iov_iter *iter,
snapc, snapc ? snapc->seq : 0);
if (write) {
int ret2 = invalidate_inode_pages2_range(inode->i_mapping,
int ret2;
ceph_fscache_invalidate(inode, true);
ret2 = invalidate_inode_pages2_range(inode->i_mapping,
pos >> PAGE_SHIFT,
(pos + count - 1) >> PAGE_SHIFT);
if (ret2 < 0)
@ -1429,6 +1433,7 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,
if (ret < 0)
return ret;
ceph_fscache_invalidate(inode, false);
ret = invalidate_inode_pages2_range(inode->i_mapping,
pos >> PAGE_SHIFT,
(pos + count - 1) >> PAGE_SHIFT);
@ -2113,6 +2118,7 @@ static long ceph_fallocate(struct file *file, int mode,
goto unlock;
filemap_invalidate_lock(inode->i_mapping);
ceph_fscache_invalidate(inode, false);
ceph_zero_pagecache_range(inode, offset, length);
ret = ceph_zero_objects(inode, offset, length);
@ -2437,6 +2443,7 @@ static ssize_t __ceph_copy_file_range(struct file *src_file, loff_t src_off,
goto out_caps;
/* Drop dst file cached pages */
ceph_fscache_invalidate(dst_inode, false);
ret = invalidate_inode_pages2_range(dst_inode->i_mapping,
dst_off >> PAGE_SHIFT,
(dst_off + len) >> PAGE_SHIFT);

22
fs/ceph/inode.c
View File

@ -564,6 +564,8 @@ void ceph_evict_inode(struct inode *inode)
percpu_counter_dec(&mdsc->metric.total_inodes);
truncate_inode_pages_final(&inode->i_data);
if (inode->i_state & I_PINNING_FSCACHE_WB)
ceph_fscache_unuse_cookie(inode, true);
clear_inode(inode);
ceph_fscache_unregister_inode_cookie(ci);
@ -634,6 +636,12 @@ int ceph_fill_file_size(struct inode *inode, int issued,
}
i_size_write(inode, size);
inode->i_blocks = calc_inode_blocks(size);
/*
* If we're expanding, then we should be able to just update
* the existing cookie.
*/
if (size > isize)
ceph_fscache_update(inode);
ci->i_reported_size = size;
if (truncate_seq != ci->i_truncate_seq) {
dout("truncate_seq %u -> %u\n",
@ -666,10 +674,6 @@ int ceph_fill_file_size(struct inode *inode, int issued,
truncate_size);
ci->i_truncate_size = truncate_size;
}
if (queue_trunc)
ceph_fscache_invalidate(inode);
return queue_trunc;
}
@ -1053,6 +1057,8 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page,
spin_unlock(&ci->i_ceph_lock);
ceph_fscache_register_inode_cookie(inode);
if (fill_inline)
ceph_fill_inline_data(inode, locked_page,
iinfo->inline_data, iinfo->inline_len);
@ -1814,11 +1820,13 @@ bool ceph_inode_set_size(struct inode *inode, loff_t size)
spin_lock(&ci->i_ceph_lock);
dout("set_size %p %llu -> %llu\n", inode, i_size_read(inode), size);
i_size_write(inode, size);
ceph_fscache_update(inode);
inode->i_blocks = calc_inode_blocks(size);
ret = __ceph_should_report_size(ci);
spin_unlock(&ci->i_ceph_lock);
return ret;
}
@ -1844,6 +1852,8 @@ static void ceph_do_invalidate_pages(struct inode *inode)
u32 orig_gen;
int check = 0;
ceph_fscache_invalidate(inode, false);
mutex_lock(&ci->i_truncate_mutex);
if (ceph_inode_is_shutdown(inode)) {
@ -1868,7 +1878,7 @@ static void ceph_do_invalidate_pages(struct inode *inode)
orig_gen = ci->i_rdcache_gen;
spin_unlock(&ci->i_ceph_lock);
ceph_fscache_invalidate(inode);
ceph_fscache_invalidate(inode, false);
if (invalidate_inode_pages2(inode->i_mapping) < 0) {
pr_err("invalidate_inode_pages2 %llx.%llx failed\n",
ceph_vinop(inode));
@ -1937,6 +1947,7 @@ void __ceph_do_pending_vmtruncate(struct inode *inode)
ci->i_truncate_pending, to);
spin_unlock(&ci->i_ceph_lock);
ceph_fscache_resize(inode, to);
truncate_pagecache(inode, to);
spin_lock(&ci->i_ceph_lock);
@ -2184,7 +2195,6 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)
if (inode_dirty_flags)
__mark_inode_dirty(inode, inode_dirty_flags);
if (mask) {
req->r_inode = inode;
ihold(inode);

10
fs/ceph/super.c
View File

@ -787,16 +787,10 @@ static int __init init_caches(void)
if (!ceph_wb_pagevec_pool)
goto bad_pagevec_pool;
error = ceph_fscache_register();
if (error)
goto bad_fscache;
return 0;
bad_fscache:
kmem_cache_destroy(ceph_mds_request_cachep);
bad_pagevec_pool:
mempool_destroy(ceph_wb_pagevec_pool);
kmem_cache_destroy(ceph_mds_request_cachep);
bad_mds_req:
kmem_cache_destroy(ceph_dir_file_cachep);
bad_dir_file:
@ -828,8 +822,6 @@ static void destroy_caches(void)
kmem_cache_destroy(ceph_dir_file_cachep);
kmem_cache_destroy(ceph_mds_request_cachep);
mempool_destroy(ceph_wb_pagevec_pool);
ceph_fscache_unregister();
}
static void __ceph_umount_begin(struct ceph_fs_client *fsc)

3
fs/ceph/super.h
View File

@ -21,7 +21,6 @@
#include <linux/ceph/libceph.h>
#ifdef CONFIG_CEPH_FSCACHE
#define FSCACHE_USE_NEW_IO_API
#include <linux/fscache.h>
#endif
@ -135,7 +134,7 @@ struct ceph_fs_client {
#endif
#ifdef CONFIG_CEPH_FSCACHE
struct fscache_cookie *fscache;
struct fscache_volume *fscache;
#endif
};

2
fs/cifs/Kconfig
View File

@ -188,7 +188,7 @@ config CIFS_SMB_DIRECT
config CIFS_FSCACHE
bool "Provide CIFS client caching support"
depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
depends on CIFS=m && FSCACHE_OLD_API || CIFS=y && FSCACHE_OLD_API=y
help
Makes CIFS FS-Cache capable. Say Y here if you want your CIFS data
to be cached locally on disk through the general filesystem cache

8
fs/fs-writeback.c
View File

@ -1668,6 +1668,13 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)
if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
inode->i_state |= I_DIRTY_PAGES;
else if (unlikely(inode->i_state & I_PINNING_FSCACHE_WB)) {
if (!(inode->i_state & I_DIRTY_PAGES)) {
inode->i_state &= ~I_PINNING_FSCACHE_WB;
wbc->unpinned_fscache_wb = true;
dirty |= I_PINNING_FSCACHE_WB; /* Cause write_inode */
}
}
spin_unlock(&inode->i_lock);
@ -1677,6 +1684,7 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)
if (ret == 0)
ret = err;
}
wbc->unpinned_fscache_wb = false;
trace_writeback_single_inode(inode, wbc, nr_to_write);
return ret;
}

3
fs/fscache/Kconfig
View File

@ -38,3 +38,6 @@ config FSCACHE_DEBUG
enabled by setting bits in /sys/modules/fscache/parameter/debug.
See Documentation/filesystems/caching/fscache.rst for more information.
config FSCACHE_OLD_API
bool

6
fs/fscache/Makefile
View File

@ -6,13 +6,9 @@
fscache-y := \
cache.o \
cookie.o \
fsdef.o \
io.o \
main.o \
netfs.o \
object.o \
operation.o \
page.o
volume.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o

650
fs/fscache/cache.c
View File

@ -1,209 +1,229 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache cache handling
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include <linux/export.h>
#include <linux/slab.h>
#include "internal.h"
LIST_HEAD(fscache_cache_list);
static LIST_HEAD(fscache_caches);
DECLARE_RWSEM(fscache_addremove_sem);
DECLARE_WAIT_QUEUE_HEAD(fscache_cache_cleared_wq);
EXPORT_SYMBOL(fscache_cache_cleared_wq);
EXPORT_SYMBOL(fscache_addremove_sem);
DECLARE_WAIT_QUEUE_HEAD(fscache_clearance_waiters);
EXPORT_SYMBOL(fscache_clearance_waiters);
static LIST_HEAD(fscache_cache_tag_list);
static atomic_t fscache_cache_debug_id;
/*
* look up a cache tag
* Allocate a cache cookie.
*/
struct fscache_cache_tag *__fscache_lookup_cache_tag(const char *name)
static struct fscache_cache *fscache_alloc_cache(const char *name)
{
struct fscache_cache_tag *tag, *xtag;
struct fscache_cache *cache;
/* firstly check for the existence of the tag under read lock */
cache = kzalloc(sizeof(*cache), GFP_KERNEL);
if (cache) {
if (name) {
cache->name = kstrdup(name, GFP_KERNEL);
if (!cache->name) {
kfree(cache);
return NULL;
}
}
refcount_set(&cache->ref, 1);
INIT_LIST_HEAD(&cache->cache_link);
cache->debug_id = atomic_inc_return(&fscache_cache_debug_id);
}
return cache;
}
static bool fscache_get_cache_maybe(struct fscache_cache *cache,
enum fscache_cache_trace where)
{
bool success;
int ref;
success = __refcount_inc_not_zero(&cache->ref, &ref);
if (success)
trace_fscache_cache(cache->debug_id, ref + 1, where);
return success;
}
/*
* Look up a cache cookie.
*/
struct fscache_cache *fscache_lookup_cache(const char *name, bool is_cache)
{
struct fscache_cache *candidate, *cache, *unnamed = NULL;
/* firstly check for the existence of the cache under read lock */
down_read(&fscache_addremove_sem);
list_for_each_entry(tag, &fscache_cache_tag_list, link) {
if (strcmp(tag->name, name) == 0) {
atomic_inc(&tag->usage);
up_read(&fscache_addremove_sem);
return tag;
list_for_each_entry(cache, &fscache_caches, cache_link) {
if (cache->name && name && strcmp(cache->name, name) == 0 &&
fscache_get_cache_maybe(cache, fscache_cache_get_acquire))
goto got_cache_r;
if (!cache->name && !name &&
fscache_get_cache_maybe(cache, fscache_cache_get_acquire))
goto got_cache_r;
}
if (!name) {
list_for_each_entry(cache, &fscache_caches, cache_link) {
if (cache->name &&
fscache_get_cache_maybe(cache, fscache_cache_get_acquire))
goto got_cache_r;
}
}
up_read(&fscache_addremove_sem);
/* the tag does not exist - create a candidate */
xtag = kzalloc(sizeof(*xtag) + strlen(name) + 1, GFP_KERNEL);
if (!xtag)
/* return a dummy tag if out of memory */
/* the cache does not exist - create a candidate */
candidate = fscache_alloc_cache(name);
if (!candidate)
return ERR_PTR(-ENOMEM);
atomic_set(&xtag->usage, 1);
strcpy(xtag->name, name);
/* write lock, search again and add if still not present */
down_write(&fscache_addremove_sem);
list_for_each_entry(tag, &fscache_cache_tag_list, link) {
if (strcmp(tag->name, name) == 0) {
atomic_inc(&tag->usage);
up_write(&fscache_addremove_sem);
kfree(xtag);
return tag;
list_for_each_entry(cache, &fscache_caches, cache_link) {
if (cache->name && name && strcmp(cache->name, name) == 0 &&
fscache_get_cache_maybe(cache, fscache_cache_get_acquire))
goto got_cache_w;
if (!cache->name) {
unnamed = cache;
if (!name &&
fscache_get_cache_maybe(cache, fscache_cache_get_acquire))
goto got_cache_w;
}
}
list_add_tail(&xtag->link, &fscache_cache_tag_list);
if (unnamed && is_cache &&
fscache_get_cache_maybe(unnamed, fscache_cache_get_acquire))
goto use_unnamed_cache;
if (!name) {
list_for_each_entry(cache, &fscache_caches, cache_link) {
if (cache->name &&
fscache_get_cache_maybe(cache, fscache_cache_get_acquire))
goto got_cache_w;
}
}
list_add_tail(&candidate->cache_link, &fscache_caches);
trace_fscache_cache(candidate->debug_id,
refcount_read(&candidate->ref),
fscache_cache_new_acquire);
up_write(&fscache_addremove_sem);
return xtag;
}
return candidate;
/*
* release a reference to a cache tag
*/
void __fscache_release_cache_tag(struct fscache_cache_tag *tag)
{
if (tag != ERR_PTR(-ENOMEM)) {
down_write(&fscache_addremove_sem);
if (atomic_dec_and_test(&tag->usage))
list_del_init(&tag->link);
else
tag = NULL;
up_write(&fscache_addremove_sem);
kfree(tag);
}
}
/*
* select a cache in which to store an object
* - the cache addremove semaphore must be at least read-locked by the caller
* - the object will never be an index
*/
struct fscache_cache *fscache_select_cache_for_object(
struct fscache_cookie *cookie)
{
struct fscache_cache_tag *tag;
struct fscache_object *object;
struct fscache_cache *cache;
_enter("");
if (list_empty(&fscache_cache_list)) {
_leave(" = NULL [no cache]");
return NULL;
}
/* we check the parent to determine the cache to use */
spin_lock(&cookie->lock);
/* the first in the parent's backing list should be the preferred
* cache */
if (!hlist_empty(&cookie->backing_objects)) {
object = hlist_entry(cookie->backing_objects.first,
struct fscache_object, cookie_link);
cache = object->cache;
if (fscache_object_is_dying(object) ||
test_bit(FSCACHE_IOERROR, &cache->flags))
cache = NULL;
spin_unlock(&cookie->lock);
_leave(" = %s [parent]", cache ? cache->tag->name : "NULL");
return cache;
}
/* the parent is unbacked */
if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX) {
/* cookie not an index and is unbacked */
spin_unlock(&cookie->lock);
_leave(" = NULL [cookie ub,ni]");
return NULL;
}
spin_unlock(&cookie->lock);
if (!cookie->def->select_cache)
goto no_preference;
/* ask the netfs for its preference */
tag = cookie->def->select_cache(cookie->parent->netfs_data,
cookie->netfs_data);
if (!tag)
goto no_preference;
if (tag == ERR_PTR(-ENOMEM)) {
_leave(" = NULL [nomem tag]");
return NULL;
}
if (!tag->cache) {
_leave(" = NULL [unbacked tag]");
return NULL;
}
if (test_bit(FSCACHE_IOERROR, &tag->cache->flags))
return NULL;
_leave(" = %s [specific]", tag->name);
return tag->cache;
no_preference:
/* netfs has no preference - just select first cache */
cache = list_entry(fscache_cache_list.next,
struct fscache_cache, link);
_leave(" = %s [first]", cache->tag->name);
got_cache_r:
up_read(&fscache_addremove_sem);
return cache;
use_unnamed_cache:
cache = unnamed;
cache->name = candidate->name;
candidate->name = NULL;
got_cache_w:
up_write(&fscache_addremove_sem);
kfree(candidate->name);
kfree(candidate);
return cache;
}
/**
* fscache_init_cache - Initialise a cache record
* @cache: The cache record to be initialised
* @ops: The cache operations to be installed in that record
* @idfmt: Format string to define identifier
* @...: sprintf-style arguments
* fscache_acquire_cache - Acquire a cache-level cookie.
* @name: The name of the cache.
*
* Initialise a record of a cache and fill in the name.
* Get a cookie to represent an actual cache. If a name is given and there is
* a nameless cache record available, this will acquire that and set its name,
* directing all the volumes using it to this cache.
*
* See Documentation/filesystems/caching/backend-api.rst for a complete
* description.
* The cache will be switched over to the preparing state if not currently in
* use, otherwise -EBUSY will be returned.
*/
void fscache_init_cache(struct fscache_cache *cache,
const struct fscache_cache_ops *ops,
const char *idfmt,
...)
struct fscache_cache *fscache_acquire_cache(const char *name)
{
va_list va;
struct fscache_cache *cache;
memset(cache, 0, sizeof(*cache));
ASSERT(name);
cache = fscache_lookup_cache(name, true);
if (IS_ERR(cache))
return cache;
cache->ops = ops;
if (!fscache_set_cache_state_maybe(cache,
FSCACHE_CACHE_IS_NOT_PRESENT,
FSCACHE_CACHE_IS_PREPARING)) {
pr_warn("Cache tag %s in use\n", name);
fscache_put_cache(cache, fscache_cache_put_cache);
return ERR_PTR(-EBUSY);
}
va_start(va, idfmt);
vsnprintf(cache->identifier, sizeof(cache->identifier), idfmt, va);
va_end(va);
INIT_WORK(&cache->op_gc, fscache_operation_gc);
INIT_LIST_HEAD(&cache->link);
INIT_LIST_HEAD(&cache->object_list);
INIT_LIST_HEAD(&cache->op_gc_list);
spin_lock_init(&cache->object_list_lock);
spin_lock_init(&cache->op_gc_list_lock);
return cache;
}
EXPORT_SYMBOL(fscache_init_cache);
EXPORT_SYMBOL(fscache_acquire_cache);
/**
* fscache_put_cache - Release a cache-level cookie.
* @cache: The cache cookie to be released
* @where: An indication of where the release happened
*
* Release the caller's reference on a cache-level cookie. The @where
* indication should give information about the circumstances in which the call
* occurs and will be logged through a tracepoint.
*/
void fscache_put_cache(struct fscache_cache *cache,
enum fscache_cache_trace where)
{
unsigned int debug_id = cache->debug_id;
bool zero;
int ref;
if (IS_ERR_OR_NULL(cache))
return;
zero = __refcount_dec_and_test(&cache->ref, &ref);
trace_fscache_cache(debug_id, ref - 1, where);
if (zero) {
down_write(&fscache_addremove_sem);
list_del_init(&cache->cache_link);
up_write(&fscache_addremove_sem);
kfree(cache->name);
kfree(cache);
}
}
/**
* fscache_relinquish_cache - Reset cache state and release cookie
* @cache: The cache cookie to be released
*
* Reset the state of a cache and release the caller's reference on a cache
* cookie.
*/
void fscache_relinquish_cache(struct fscache_cache *cache)
{
enum fscache_cache_trace where =
(cache->state == FSCACHE_CACHE_IS_PREPARING) ?
fscache_cache_put_prep_failed :
fscache_cache_put_relinquish;
cache->ops = NULL;
cache->cache_priv = NULL;
smp_store_release(&cache->state, FSCACHE_CACHE_IS_NOT_PRESENT);
fscache_put_cache(cache, where);
}
EXPORT_SYMBOL(fscache_relinquish_cache);
/**
* fscache_add_cache - Declare a cache as being open for business
* @cache: The record describing the cache
* @ifsdef: The record of the cache object describing the top-level index
* @tagname: The tag describing this cache
* @cache: The cache-level cookie representing the cache
* @ops: Table of cache operations to use
* @cache_priv: Private data for the cache record
*
* Add a cache to the system, making it available for netfs's to use.
*
@ -211,94 +231,98 @@ EXPORT_SYMBOL(fscache_init_cache);
* description.
*/
int fscache_add_cache(struct fscache_cache *cache,
struct fscache_object *ifsdef,
const char *tagname)
const struct fscache_cache_ops *ops,
void *cache_priv)
{
struct fscache_cache_tag *tag;
int n_accesses;
ASSERTCMP(ifsdef->cookie, ==, &fscache_fsdef_index);
BUG_ON(!cache->ops);
BUG_ON(!ifsdef);
_enter("{%s,%s}", ops->name, cache->name);
cache->flags = 0;
ifsdef->event_mask =
((1 << NR_FSCACHE_OBJECT_EVENTS) - 1) &
~(1 << FSCACHE_OBJECT_EV_CLEARED);
__set_bit(FSCACHE_OBJECT_IS_AVAILABLE, &ifsdef->flags);
BUG_ON(fscache_cache_state(cache) != FSCACHE_CACHE_IS_PREPARING);
if (!tagname)
tagname = cache->identifier;
BUG_ON(!tagname[0]);
_enter("{%s.%s},,%s", cache->ops->name, cache->identifier, tagname);
/* we use the cache tag to uniquely identify caches */
tag = __fscache_lookup_cache_tag(tagname);
if (IS_ERR(tag))
goto nomem;
if (test_and_set_bit(FSCACHE_TAG_RESERVED, &tag->flags))
goto tag_in_use;
cache->kobj = kobject_create_and_add(tagname, fscache_root);
if (!cache->kobj)
goto error;
ifsdef->cache = cache;
cache->fsdef = ifsdef;
/* Get a ref on the cache cookie and keep its n_accesses counter raised
* by 1 to prevent wakeups from transitioning it to 0 until we're
* withdrawing caching services from it.
*/
n_accesses = atomic_inc_return(&cache->n_accesses);
trace_fscache_access_cache(cache->debug_id, refcount_read(&cache->ref),
n_accesses, fscache_access_cache_pin);
down_write(&fscache_addremove_sem);
tag->cache = cache;
cache->tag = tag;
cache->ops = ops;
cache->cache_priv = cache_priv;
fscache_set_cache_state(cache, FSCACHE_CACHE_IS_ACTIVE);
/* add the cache to the list */
list_add(&cache->link, &fscache_cache_list);
/* add the cache's netfs definition index object to the cache's
* list */
spin_lock(&cache->object_list_lock);
list_add_tail(&ifsdef->cache_link, &cache->object_list);
spin_unlock(&cache->object_list_lock);
/* add the cache's netfs definition index object to the top level index
* cookie as a known backing object */
spin_lock(&fscache_fsdef_index.lock);
hlist_add_head(&ifsdef->cookie_link,
&fscache_fsdef_index.backing_objects);
refcount_inc(&fscache_fsdef_index.ref);
/* done */
spin_unlock(&fscache_fsdef_index.lock);
up_write(&fscache_addremove_sem);
pr_notice("Cache \"%s\" added (type %s)\n",
cache->tag->name, cache->ops->name);
kobject_uevent(cache->kobj, KOBJ_ADD);
_leave(" = 0 [%s]", cache->identifier);
pr_notice("Cache \"%s\" added (type %s)\n", cache->name, ops->name);
_leave(" = 0 [%s]", cache->name);
return 0;
tag_in_use:
pr_err("Cache tag '%s' already in use\n", tagname);
__fscache_release_cache_tag(tag);
_leave(" = -EXIST");
return -EEXIST;
error:
__fscache_release_cache_tag(tag);
_leave(" = -EINVAL");
return -EINVAL;
nomem:
_leave(" = -ENOMEM");
return -ENOMEM;
}
EXPORT_SYMBOL(fscache_add_cache);
/**
* fscache_begin_cache_access - Pin a cache so it can be accessed
* @cache: The cache-level cookie
* @why: An indication of the circumstances of the access for tracing
*
* Attempt to pin the cache to prevent it from going away whilst we're
* accessing it and returns true if successful. This works as follows:
*
* (1) If the cache tests as not live (state is not FSCACHE_CACHE_IS_ACTIVE),
* then we return false to indicate access was not permitted.
*
* (2) If the cache tests as live, then we increment the n_accesses count and
* then recheck the liveness, ending the access if it ceased to be live.
*
* (3) When we end the access, we decrement n_accesses and wake up the any
* waiters if it reaches 0.
*
* (4) Whilst the cache is caching, n_accesses is kept artificially
* incremented to prevent wakeups from happening.
*
* (5) When the cache is taken offline, the state is changed to prevent new
* accesses, n_accesses is decremented and we wait for n_accesses to
* become 0.
*/
bool fscache_begin_cache_access(struct fscache_cache *cache, enum fscache_access_trace why)
{
int n_accesses;
if (!fscache_cache_is_live(cache))
return false;
n_accesses = atomic_inc_return(&cache->n_accesses);
smp_mb__after_atomic(); /* Reread live flag after n_accesses */
trace_fscache_access_cache(cache->debug_id, refcount_read(&cache->ref),
n_accesses, why);
if (!fscache_cache_is_live(cache)) {
fscache_end_cache_access(cache, fscache_access_unlive);
return false;
}
return true;
}
/**
* fscache_end_cache_access - Unpin a cache at the end of an access.
* @cache: The cache-level cookie
* @why: An indication of the circumstances of the access for tracing
*
* Unpin a cache after we've accessed it. The @why indicator is merely
* provided for tracing purposes.
*/
void fscache_end_cache_access(struct fscache_cache *cache, enum fscache_access_trace why)
{
int n_accesses;
smp_mb__before_atomic();
n_accesses = atomic_dec_return(&cache->n_accesses);
trace_fscache_access_cache(cache->debug_id, refcount_read(&cache->ref),
n_accesses, why);
if (n_accesses == 0)
wake_up_var(&cache->n_accesses);
}
/**
* fscache_io_error - Note a cache I/O error
* @cache: The record describing the cache
@ -311,106 +335,94 @@ EXPORT_SYMBOL(fscache_add_cache);
*/
void fscache_io_error(struct fscache_cache *cache)
{
if (!test_and_set_bit(FSCACHE_IOERROR, &cache->flags))
if (fscache_set_cache_state_maybe(cache,
FSCACHE_CACHE_IS_ACTIVE,
FSCACHE_CACHE_GOT_IOERROR))
pr_err("Cache '%s' stopped due to I/O error\n",
cache->ops->name);
cache->name);
}
EXPORT_SYMBOL(fscache_io_error);
/*
* request withdrawal of all the objects in a cache
* - all the objects being withdrawn are moved onto the supplied list
*/
static void fscache_withdraw_all_objects(struct fscache_cache *cache,
struct list_head *dying_objects)
{
struct fscache_object *object;
while (!list_empty(&cache->object_list)) {
spin_lock(&cache->object_list_lock);
if (!list_empty(&cache->object_list)) {
object = list_entry(cache->object_list.next,
struct fscache_object, cache_link);
list_move_tail(&object->cache_link, dying_objects);
_debug("withdraw %x", object->cookie->debug_id);
/* This must be done under object_list_lock to prevent
* a race with fscache_drop_object().
*/
fscache_raise_event(object, FSCACHE_OBJECT_EV_KILL);
}
spin_unlock(&cache->object_list_lock);
cond_resched();
}
}
/**
* fscache_withdraw_cache - Withdraw a cache from the active service
* @cache: The record describing the cache
* @cache: The cache cookie
*
* Withdraw a cache from service, unbinding all its cache objects from the
* netfs cookies they're currently representing.
*
* See Documentation/filesystems/caching/backend-api.rst for a complete
* description.
* Begin the process of withdrawing a cache from service. This stops new
* cache-level and volume-level accesses from taking place and waits for
* currently ongoing cache-level accesses to end.
*/
void fscache_withdraw_cache(struct fscache_cache *cache)
{
LIST_HEAD(dying_objects);
int n_accesses;
_enter("");
pr_notice("Withdrawing cache \"%s\" (%u objs)\n",
cache->name, atomic_read(&cache->object_count));
pr_notice("Withdrawing cache \"%s\"\n",
cache->tag->name);
fscache_set_cache_state(cache, FSCACHE_CACHE_IS_WITHDRAWN);
/* make the cache unavailable for cookie acquisition */
if (test_and_set_bit(FSCACHE_CACHE_WITHDRAWN, &cache->flags))
BUG();
/* Allow wakeups on dec-to-0 */
n_accesses = atomic_dec_return(&cache->n_accesses);
trace_fscache_access_cache(cache->debug_id, refcount_read(&cache->ref),
n_accesses, fscache_access_cache_unpin);
down_write(&fscache_addremove_sem);
list_del_init(&cache->link);
cache->tag->cache = NULL;
up_write(&fscache_addremove_sem);
/* make sure all pages pinned by operations on behalf of the netfs are
* written to disk */
fscache_stat(&fscache_n_cop_sync_cache);
cache->ops->sync_cache(cache);
fscache_stat_d(&fscache_n_cop_sync_cache);
/* dissociate all the netfs pages backed by this cache from the block
* mappings in the cache */
fscache_stat(&fscache_n_cop_dissociate_pages);
cache->ops->dissociate_pages(cache);
fscache_stat_d(&fscache_n_cop_dissociate_pages);
/* we now have to destroy all the active objects pertaining to this
* cache - which we do by passing them off to thread pool to be
* disposed of */
_debug("destroy");
fscache_withdraw_all_objects(cache, &dying_objects);
/* wait for all extant objects to finish their outstanding operations
* and go away */
_debug("wait for finish");
wait_event(fscache_cache_cleared_wq,
atomic_read(&cache->object_count) == 0);
_debug("wait for clearance");
wait_event(fscache_cache_cleared_wq,
list_empty(&cache->object_list));
_debug("cleared");
ASSERT(list_empty(&dying_objects));
kobject_put(cache->kobj);
clear_bit(FSCACHE_TAG_RESERVED, &cache->tag->flags);
fscache_release_cache_tag(cache->tag);
cache->tag = NULL;
_leave("");
wait_var_event(&cache->n_accesses,
atomic_read(&cache->n_accesses) == 0);
}
EXPORT_SYMBOL(fscache_withdraw_cache);
#ifdef CONFIG_PROC_FS
static const char fscache_cache_states[NR__FSCACHE_CACHE_STATE] = "-PAEW";
/*
* Generate a list of caches in /proc/fs/fscache/caches
*/
static int fscache_caches_seq_show(struct seq_file *m, void *v)
{
struct fscache_cache *cache;
if (v == &fscache_caches) {
seq_puts(m,
"CACHE REF VOLS OBJS ACCES S NAME\n"
"======== ===== ===== ===== ===== = ===============\n"
);
return 0;
}
cache = list_entry(v, struct fscache_cache, cache_link);
seq_printf(m,
"%08x %5d %5d %5d %5d %c %s\n",
cache->debug_id,
refcount_read(&cache->ref),
atomic_read(&cache->n_volumes),
atomic_read(&cache->object_count),
atomic_read(&cache->n_accesses),
fscache_cache_states[cache->state],
cache->name ?: "-");
return 0;
}
static void *fscache_caches_seq_start(struct seq_file *m, loff_t *_pos)
__acquires(fscache_addremove_sem)
{
down_read(&fscache_addremove_sem);
return seq_list_start_head(&fscache_caches, *_pos);
}
static void *fscache_caches_seq_next(struct seq_file *m, void *v, loff_t *_pos)
{
return seq_list_next(v, &fscache_caches, _pos);
}
static void fscache_caches_seq_stop(struct seq_file *m, void *v)
__releases(fscache_addremove_sem)
{
up_read(&fscache_addremove_sem);
}
const struct seq_operations fscache_caches_seq_ops = {
.start = fscache_caches_seq_start,
.next = fscache_caches_seq_next,
.stop = fscache_caches_seq_stop,
.show = fscache_caches_seq_show,
};
#endif /* CONFIG_PROC_FS */

1584
fs/fscache/cookie.c
View File

File diff suppressed because it is too large Load Diff

98
fs/fscache/fsdef.c
View File

@ -1,98 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Filesystem index definition
*
* Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include "internal.h"
static
enum fscache_checkaux fscache_fsdef_netfs_check_aux(void *cookie_netfs_data,
const void *data,
uint16_t datalen,
loff_t object_size);
/*
* The root index is owned by FS-Cache itself.
*
* When a netfs requests caching facilities, FS-Cache will, if one doesn't
* already exist, create an entry in the root index with the key being the name
* of the netfs ("AFS" for example), and the auxiliary data holding the index
* structure version supplied by the netfs:
*
* FSDEF
* |
* +-----------+
* | |
* NFS AFS
* [v=1] [v=1]
*
* If an entry with the appropriate name does already exist, the version is
* compared. If the version is different, the entire subtree from that entry
* will be discarded and a new entry created.
*
* The new entry will be an index, and a cookie referring to it will be passed
* to the netfs. This is then the root handle by which the netfs accesses the
* cache. It can create whatever objects it likes in that index, including
* further indices.
*/
static struct fscache_cookie_def fscache_fsdef_index_def = {
.name = ".FS-Cache",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
struct fscache_cookie fscache_fsdef_index = {
.debug_id = 1,
.ref = REFCOUNT_INIT(1),
.n_active = ATOMIC_INIT(1),
.lock = __SPIN_LOCK_UNLOCKED(fscache_fsdef_index.lock),
.backing_objects = HLIST_HEAD_INIT,
.def = &fscache_fsdef_index_def,
.flags = 1 << FSCACHE_COOKIE_ENABLED,
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
EXPORT_SYMBOL(fscache_fsdef_index);
/*
* Definition of an entry in the root index. Each entry is an index, keyed to
* a specific netfs and only applicable to a particular version of the index
* structure used by that netfs.
*/
struct fscache_cookie_def fscache_fsdef_netfs_def = {
.name = "FSDEF.netfs",
.type = FSCACHE_COOKIE_TYPE_INDEX,
.check_aux = fscache_fsdef_netfs_check_aux,
};
/*
* check that the index structure version number stored in the auxiliary data
* matches the one the netfs gave us
*/
static enum fscache_checkaux fscache_fsdef_netfs_check_aux(
void *cookie_netfs_data,
const void *data,
uint16_t datalen,
loff_t object_size)
{
struct fscache_netfs *netfs = cookie_netfs_data;
uint32_t version;
_enter("{%s},,%hu", netfs->name, datalen);
if (datalen != sizeof(version)) {
_leave(" = OBSOLETE [dl=%d v=%zu]", datalen, sizeof(version));
return FSCACHE_CHECKAUX_OBSOLETE;
}
memcpy(&version, data, sizeof(version));
if (version != netfs->version) {
_leave(" = OBSOLETE [ver=%x net=%x]", version, netfs->version);
return FSCACHE_CHECKAUX_OBSOLETE;
}
_leave(" = OKAY");
return FSCACHE_CHECKAUX_OKAY;
}

315
fs/fscache/internal.h
View File

@ -1,65 +1,69 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/* Internal definitions for FS-Cache
*
* Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
/*
* Lock order, in the order in which multiple locks should be obtained:
* - fscache_addremove_sem
* - cookie->lock
* - cookie->parent->lock
* - cache->object_list_lock
* - object->lock
* - object->parent->lock
* - cookie->stores_lock
* - fscache_thread_lock
*
*/
#ifdef pr_fmt
#undef pr_fmt
#endif
#define pr_fmt(fmt) "FS-Cache: " fmt
#include <linux/slab.h>
#include <linux/fscache-cache.h>
#include <trace/events/fscache.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
#define FSCACHE_MIN_THREADS 4
#define FSCACHE_MAX_THREADS 32
/*
* cache.c
*/
extern struct list_head fscache_cache_list;
extern struct rw_semaphore fscache_addremove_sem;
#ifdef CONFIG_PROC_FS
extern const struct seq_operations fscache_caches_seq_ops;
#endif
bool fscache_begin_cache_access(struct fscache_cache *cache, enum fscache_access_trace why);
void fscache_end_cache_access(struct fscache_cache *cache, enum fscache_access_trace why);
struct fscache_cache *fscache_lookup_cache(const char *name, bool is_cache);
void fscache_put_cache(struct fscache_cache *cache, enum fscache_cache_trace where);
extern struct fscache_cache *fscache_select_cache_for_object(
struct fscache_cookie *);
static inline enum fscache_cache_state fscache_cache_state(const struct fscache_cache *cache)
{
return smp_load_acquire(&cache->state);
}
static inline bool fscache_cache_is_live(const struct fscache_cache *cache)
{
return fscache_cache_state(cache) == FSCACHE_CACHE_IS_ACTIVE;
}
static inline void fscache_set_cache_state(struct fscache_cache *cache,
enum fscache_cache_state new_state)
{
smp_store_release(&cache->state, new_state);
}
static inline bool fscache_set_cache_state_maybe(struct fscache_cache *cache,
enum fscache_cache_state old_state,
enum fscache_cache_state new_state)
{
return try_cmpxchg_release(&cache->state, &old_state, new_state);
}
/*
* cookie.c
*/
extern struct kmem_cache *fscache_cookie_jar;
extern const struct seq_operations fscache_cookies_seq_ops;
extern struct timer_list fscache_cookie_lru_timer;
extern void fscache_free_cookie(struct fscache_cookie *);
extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
const struct fscache_cookie_def *,
const void *, size_t,
const void *, size_t,
void *, loff_t);
extern struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *);
extern struct fscache_cookie *fscache_cookie_get(struct fscache_cookie *,
enum fscache_cookie_trace);
extern void fscache_cookie_put(struct fscache_cookie *,
enum fscache_cookie_trace);
extern void fscache_print_cookie(struct fscache_cookie *cookie, char prefix);
extern bool fscache_begin_cookie_access(struct fscache_cookie *cookie,
enum fscache_access_trace why);
static inline void fscache_cookie_see(struct fscache_cookie *cookie,
static inline void fscache_see_cookie(struct fscache_cookie *cookie,
enum fscache_cookie_trace where)
{
trace_fscache_cookie(cookie->debug_id, refcount_read(&cookie->ref),
@ -67,60 +71,22 @@ static inline void fscache_cookie_see(struct fscache_cookie *cookie,
}
/*
* fsdef.c
* io.c
*/
extern struct fscache_cookie fscache_fsdef_index;
extern struct fscache_cookie_def fscache_fsdef_netfs_def;
static inline void fscache_end_operation(struct netfs_cache_resources *cres)
{
const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
if (ops)
ops->end_operation(cres);
}
/*
* main.c
*/
extern unsigned fscache_defer_lookup;
extern unsigned fscache_defer_create;
extern unsigned fscache_debug;
extern struct kobject *fscache_root;
extern struct workqueue_struct *fscache_object_wq;
extern struct workqueue_struct *fscache_op_wq;
DECLARE_PER_CPU(wait_queue_head_t, fscache_object_cong_wait);
extern unsigned int fscache_hash(unsigned int salt, unsigned int *data, unsigned int n);
static inline bool fscache_object_congested(void)
{
return workqueue_congested(WORK_CPU_UNBOUND, fscache_object_wq);
}
/*
* object.c
*/
extern void fscache_enqueue_object(struct fscache_object *);
/*
* operation.c
*/
extern int fscache_submit_exclusive_op(struct fscache_object *,
struct fscache_operation *);
extern int fscache_submit_op(struct fscache_object *,
struct fscache_operation *);
extern int fscache_cancel_op(struct fscache_operation *, bool);
extern void fscache_cancel_all_ops(struct fscache_object *);
extern void fscache_abort_object(struct fscache_object *);
extern void fscache_start_operations(struct fscache_object *);
extern void fscache_operation_gc(struct work_struct *);
/*
* page.c
*/
extern int fscache_wait_for_deferred_lookup(struct fscache_cookie *);
extern int fscache_wait_for_operation_activation(struct fscache_object *,
struct fscache_operation *,
atomic_t *,
atomic_t *);
extern void fscache_invalidate_writes(struct fscache_cookie *);
struct fscache_retrieval *fscache_alloc_retrieval(struct fscache_cookie *cookie,
struct address_space *mapping,
fscache_rw_complete_t end_io_func,
void *context);
extern unsigned int fscache_hash(unsigned int salt, const void *data, size_t len);
/*
* proc.c
@ -137,125 +103,27 @@ extern void fscache_proc_cleanup(void);
* stats.c
*/
#ifdef CONFIG_FSCACHE_STATS
extern atomic_t fscache_n_ops_processed[FSCACHE_MAX_THREADS];
extern atomic_t fscache_n_objs_processed[FSCACHE_MAX_THREADS];
extern atomic_t fscache_n_op_pend;
extern atomic_t fscache_n_op_run;
extern atomic_t fscache_n_op_enqueue;
extern atomic_t fscache_n_op_deferred_release;
extern atomic_t fscache_n_op_initialised;
extern atomic_t fscache_n_op_release;
extern atomic_t fscache_n_op_gc;
extern atomic_t fscache_n_op_cancelled;
extern atomic_t fscache_n_op_rejected;
extern atomic_t fscache_n_attr_changed;
extern atomic_t fscache_n_attr_changed_ok;
extern atomic_t fscache_n_attr_changed_nobufs;
extern atomic_t fscache_n_attr_changed_nomem;
extern atomic_t fscache_n_attr_changed_calls;
extern atomic_t fscache_n_allocs;
extern atomic_t fscache_n_allocs_ok;
extern atomic_t fscache_n_allocs_wait;
extern atomic_t fscache_n_allocs_nobufs;
extern atomic_t fscache_n_allocs_intr;
extern atomic_t fscache_n_allocs_object_dead;
extern atomic_t fscache_n_alloc_ops;
extern atomic_t fscache_n_alloc_op_waits;
extern atomic_t fscache_n_retrievals;
extern atomic_t fscache_n_retrievals_ok;
extern atomic_t fscache_n_retrievals_wait;
extern atomic_t fscache_n_retrievals_nodata;
extern atomic_t fscache_n_retrievals_nobufs;
extern atomic_t fscache_n_retrievals_intr;
extern atomic_t fscache_n_retrievals_nomem;
extern atomic_t fscache_n_retrievals_object_dead;
extern atomic_t fscache_n_retrieval_ops;
extern atomic_t fscache_n_retrieval_op_waits;
extern atomic_t fscache_n_stores;
extern atomic_t fscache_n_stores_ok;
extern atomic_t fscache_n_stores_again;
extern atomic_t fscache_n_stores_nobufs;
extern atomic_t fscache_n_stores_oom;
extern atomic_t fscache_n_store_ops;
extern atomic_t fscache_n_store_calls;
extern atomic_t fscache_n_store_pages;
extern atomic_t fscache_n_store_radix_deletes;
extern atomic_t fscache_n_store_pages_over_limit;
extern atomic_t fscache_n_store_vmscan_not_storing;
extern atomic_t fscache_n_store_vmscan_gone;
extern atomic_t fscache_n_store_vmscan_busy;
extern atomic_t fscache_n_store_vmscan_cancelled;
extern atomic_t fscache_n_store_vmscan_wait;
extern atomic_t fscache_n_marks;
extern atomic_t fscache_n_uncaches;
extern atomic_t fscache_n_volumes;
extern atomic_t fscache_n_volumes_collision;
extern atomic_t fscache_n_volumes_nomem;
extern atomic_t fscache_n_cookies;
extern atomic_t fscache_n_cookies_lru;
extern atomic_t fscache_n_cookies_lru_expired;
extern atomic_t fscache_n_cookies_lru_removed;
extern atomic_t fscache_n_cookies_lru_dropped;
extern atomic_t fscache_n_acquires;
extern atomic_t fscache_n_acquires_null;
extern atomic_t fscache_n_acquires_no_cache;
extern atomic_t fscache_n_acquires_ok;
extern atomic_t fscache_n_acquires_nobufs;
extern atomic_t fscache_n_acquires_oom;
extern atomic_t fscache_n_invalidates;
extern atomic_t fscache_n_invalidates_run;
extern atomic_t fscache_n_updates;
extern atomic_t fscache_n_updates_null;
extern atomic_t fscache_n_updates_run;
extern atomic_t fscache_n_relinquishes;
extern atomic_t fscache_n_relinquishes_null;
extern atomic_t fscache_n_relinquishes_waitcrt;
extern atomic_t fscache_n_relinquishes_retire;
extern atomic_t fscache_n_relinquishes_dropped;
extern atomic_t fscache_n_cookie_index;
extern atomic_t fscache_n_cookie_data;
extern atomic_t fscache_n_cookie_special;
extern atomic_t fscache_n_object_alloc;
extern atomic_t fscache_n_object_no_alloc;
extern atomic_t fscache_n_object_lookups;
extern atomic_t fscache_n_object_lookups_negative;
extern atomic_t fscache_n_object_lookups_positive;
extern atomic_t fscache_n_object_lookups_timed_out;
extern atomic_t fscache_n_object_created;
extern atomic_t fscache_n_object_avail;
extern atomic_t fscache_n_object_dead;
extern atomic_t fscache_n_checkaux_none;
extern atomic_t fscache_n_checkaux_okay;
extern atomic_t fscache_n_checkaux_update;
extern atomic_t fscache_n_checkaux_obsolete;
extern atomic_t fscache_n_cop_alloc_object;
extern atomic_t fscache_n_cop_lookup_object;
extern atomic_t fscache_n_cop_lookup_complete;
extern atomic_t fscache_n_cop_grab_object;
extern atomic_t fscache_n_cop_invalidate_object;
extern atomic_t fscache_n_cop_update_object;
extern atomic_t fscache_n_cop_drop_object;
extern atomic_t fscache_n_cop_put_object;
extern atomic_t fscache_n_cop_sync_cache;
extern atomic_t fscache_n_cop_attr_changed;
extern atomic_t fscache_n_cop_read_or_alloc_page;
extern atomic_t fscache_n_cop_read_or_alloc_pages;
extern atomic_t fscache_n_cop_allocate_page;
extern atomic_t fscache_n_cop_allocate_pages;
extern atomic_t fscache_n_cop_write_page;
extern atomic_t fscache_n_cop_uncache_page;
extern atomic_t fscache_n_cop_dissociate_pages;
extern atomic_t fscache_n_cache_no_space_reject;
extern atomic_t fscache_n_cache_stale_objects;
extern atomic_t fscache_n_cache_retired_objects;
extern atomic_t fscache_n_cache_culled_objects;
extern atomic_t fscache_n_resizes;
extern atomic_t fscache_n_resizes_null;
static inline void fscache_stat(atomic_t *stat)
{
@ -278,71 +146,26 @@ int fscache_stats_show(struct seq_file *m, void *v);
#endif
/*
* raise an event on an object
* - if the event is not masked for that object, then the object is
* queued for attention by the thread pool.
* volume.c
*/
static inline void fscache_raise_event(struct fscache_object *object,
unsigned event)
{
BUG_ON(event >= NR_FSCACHE_OBJECT_EVENTS);
#if 0
printk("*** fscache_raise_event(OBJ%d{%lx},%x)\n",
object->debug_id, object->event_mask, (1 << event));
#endif
if (!test_and_set_bit(event, &object->events) &&
test_bit(event, &object->event_mask))
fscache_enqueue_object(object);
}
extern const struct seq_operations fscache_volumes_seq_ops;
/*
* get an extra reference to a netfs retrieval context
*/
static inline
void *fscache_get_context(struct fscache_cookie *cookie, void *context)
{
if (cookie->def->get_context)
cookie->def->get_context(cookie->netfs_data, context);
return context;
}
struct fscache_volume *fscache_get_volume(struct fscache_volume *volume,
enum fscache_volume_trace where);
void fscache_put_volume(struct fscache_volume *volume,
enum fscache_volume_trace where);
bool fscache_begin_volume_access(struct fscache_volume *volume,
struct fscache_cookie *cookie,
enum fscache_access_trace why);
void fscache_create_volume(struct fscache_volume *volume, bool wait);
/*
* release a reference to a netfs retrieval context
*/
static inline
void fscache_put_context(struct fscache_cookie *cookie, void *context)
{
if (cookie->def->put_context)
cookie->def->put_context(cookie->netfs_data, context);
}
/*
* Update the auxiliary data on a cookie.
*/
static inline
void fscache_update_aux(struct fscache_cookie *cookie, const void *aux_data)
{
void *p;
if (!aux_data)
return;
if (cookie->aux_len <= sizeof(cookie->inline_aux))
p = cookie->inline_aux;
else
p = cookie->aux;
if (memcmp(p, aux_data, cookie->aux_len) != 0) {
memcpy(p, aux_data, cookie->aux_len);
set_bit(FSCACHE_COOKIE_AUX_UPDATED, &cookie->flags);
}
}
/*****************************************************************************/
/*
* debug tracing
*/
#define dbgprintk(FMT, ...) \
printk(KERN_DEBUG "[%-6.6s] "FMT"\n", current->comm, ##__VA_ARGS__)
printk("[%-6.6s] "FMT"\n", current->comm, ##__VA_ARGS__)
#define kenter(FMT, ...) dbgprintk("==> %s("FMT")", __func__, ##__VA_ARGS__)
#define kleave(FMT, ...) dbgprintk("<== %s()"FMT"", __func__, ##__VA_ARGS__)
@ -395,7 +218,7 @@ do { \
#define FSCACHE_DEBUG_CACHE 0
#define FSCACHE_DEBUG_COOKIE 1
#define FSCACHE_DEBUG_PAGE 2
#define FSCACHE_DEBUG_OBJECT 2
#define FSCACHE_DEBUG_OPERATION 3
#define FSCACHE_POINT_ENTER 1

384
fs/fscache/io.c
View File

@ -4,113 +4,323 @@
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL PAGE
#include <linux/module.h>
#define FSCACHE_USE_NEW_IO_API
#define FSCACHE_DEBUG_LEVEL OPERATION
#include <linux/fscache-cache.h>
#include <linux/uio.h>
#include <linux/bvec.h>
#include <linux/slab.h>
#include <linux/netfs.h>
#include <linux/uio.h>
#include "internal.h"
/*
* Start a cache read operation.
* - we return:
* -ENOMEM - out of memory, some pages may be being read
* -ERESTARTSYS - interrupted, some pages may be being read
* -ENOBUFS - no backing object or space available in which to cache any
* pages not being read
* -ENODATA - no data available in the backing object for some or all of
* the pages
* 0 - dispatched a read on all pages
/**
* fscache_wait_for_operation - Wait for an object become accessible
* @cres: The cache resources for the operation being performed
* @want_state: The minimum state the object must be at
*
* See if the target cache object is at the specified minimum state of
* accessibility yet, and if not, wait for it.
*/
int __fscache_begin_read_operation(struct netfs_read_request *rreq,
struct fscache_cookie *cookie)
bool fscache_wait_for_operation(struct netfs_cache_resources *cres,
enum fscache_want_state want_state)
{
struct fscache_retrieval *op;
struct fscache_object *object;
bool wake_cookie = false;
int ret;
struct fscache_cookie *cookie = fscache_cres_cookie(cres);
enum fscache_cookie_state state;
_enter("rr=%08x", rreq->debug_id);
fscache_stat(&fscache_n_retrievals);
if (hlist_empty(&cookie->backing_objects))
goto nobufs;
if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
_leave(" = -ENOBUFS [invalidating]");
return -ENOBUFS;
again:
if (!fscache_cache_is_live(cookie->volume->cache)) {
_leave(" [broken]");
return false;
}
ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
state = fscache_cookie_state(cookie);
_enter("c=%08x{%u},%x", cookie->debug_id, state, want_state);
if (fscache_wait_for_deferred_lookup(cookie) < 0)
return -ERESTARTSYS;
switch (state) {
case FSCACHE_COOKIE_STATE_CREATING:
case FSCACHE_COOKIE_STATE_INVALIDATING:
if (want_state == FSCACHE_WANT_PARAMS)
goto ready; /* There can be no content */
fallthrough;
case FSCACHE_COOKIE_STATE_LOOKING_UP:
case FSCACHE_COOKIE_STATE_LRU_DISCARDING:
wait_var_event(&cookie->state,
fscache_cookie_state(cookie) != state);
goto again;
op = fscache_alloc_retrieval(cookie, NULL, NULL, NULL);
if (!op)
return -ENOMEM;
trace_fscache_page_op(cookie, NULL, &op->op, fscache_page_op_retr_multi);
case FSCACHE_COOKIE_STATE_ACTIVE:
goto ready;
case FSCACHE_COOKIE_STATE_DROPPED:
case FSCACHE_COOKIE_STATE_RELINQUISHING:
default:
_leave(" [not live]");
return false;
}
ready:
if (!cres->cache_priv2)
return cookie->volume->cache->ops->begin_operation(cres, want_state);
return true;
}
EXPORT_SYMBOL(fscache_wait_for_operation);
/*
* Begin an I/O operation on the cache, waiting till we reach the right state.
*
* Attaches the resources required to the operation resources record.
*/
static int fscache_begin_operation(struct netfs_cache_resources *cres,
struct fscache_cookie *cookie,
enum fscache_want_state want_state,
enum fscache_access_trace why)
{
enum fscache_cookie_state state;
long timeo;
bool once_only = false;
cres->ops = NULL;
cres->cache_priv = cookie;
cres->cache_priv2 = NULL;
cres->debug_id = cookie->debug_id;
cres->inval_counter = cookie->inval_counter;
if (!fscache_begin_cookie_access(cookie, why))
return -ENOBUFS;
again:
spin_lock(&cookie->lock);
if (!fscache_cookie_enabled(cookie) ||
hlist_empty(&cookie->backing_objects))
goto nobufs_unlock;
object = hlist_entry(cookie->backing_objects.first,
struct fscache_object, cookie_link);
state = fscache_cookie_state(cookie);
_enter("c=%08x{%u},%x", cookie->debug_id, state, want_state);
__fscache_use_cookie(cookie);
atomic_inc(&object->n_reads);
__set_bit(FSCACHE_OP_DEC_READ_CNT, &op->op.flags);
switch (state) {
case FSCACHE_COOKIE_STATE_LOOKING_UP:
case FSCACHE_COOKIE_STATE_LRU_DISCARDING:
case FSCACHE_COOKIE_STATE_INVALIDATING:
goto wait_for_file_wrangling;
case FSCACHE_COOKIE_STATE_CREATING:
if (want_state == FSCACHE_WANT_PARAMS)
goto ready; /* There can be no content */
goto wait_for_file_wrangling;
case FSCACHE_COOKIE_STATE_ACTIVE:
goto ready;
case FSCACHE_COOKIE_STATE_DROPPED:
case FSCACHE_COOKIE_STATE_RELINQUISHING:
WARN(1, "Can't use cookie in state %u\n", cookie->state);
goto not_live;
default:
goto not_live;
}
if (fscache_submit_op(object, &op->op) < 0)
goto nobufs_unlock_dec;
ready:
spin_unlock(&cookie->lock);
if (!cookie->volume->cache->ops->begin_operation(cres, want_state))
goto failed;
return 0;
fscache_stat(&fscache_n_retrieval_ops);
/* we wait for the operation to become active, and then process it
* *here*, in this thread, and not in the thread pool */
ret = fscache_wait_for_operation_activation(
object, &op->op,
__fscache_stat(&fscache_n_retrieval_op_waits),
__fscache_stat(&fscache_n_retrievals_object_dead));
if (ret < 0)
goto error;
/* ask the cache to honour the operation */
ret = object->cache->ops->begin_read_operation(rreq, op);
error:
if (ret == -ENOMEM)
fscache_stat(&fscache_n_retrievals_nomem);
else if (ret == -ERESTARTSYS)
fscache_stat(&fscache_n_retrievals_intr);
else if (ret == -ENODATA)
fscache_stat(&fscache_n_retrievals_nodata);
else if (ret < 0)
fscache_stat(&fscache_n_retrievals_nobufs);
else
fscache_stat(&fscache_n_retrievals_ok);
fscache_put_retrieval(op);
_leave(" = %d", ret);
return ret;
nobufs_unlock_dec:
atomic_dec(&object->n_reads);
wake_cookie = __fscache_unuse_cookie(cookie);
nobufs_unlock:
wait_for_file_wrangling:
spin_unlock(&cookie->lock);
fscache_put_retrieval(op);
if (wake_cookie)
__fscache_wake_unused_cookie(cookie);
nobufs:
fscache_stat(&fscache_n_retrievals_nobufs);
trace_fscache_access(cookie->debug_id, refcount_read(&cookie->ref),
atomic_read(&cookie->n_accesses),
fscache_access_io_wait);
timeo = wait_var_event_timeout(&cookie->state,
fscache_cookie_state(cookie) != state, 20 * HZ);
if (timeo <= 1 && !once_only) {
pr_warn("%s: cookie state change wait timed out: cookie->state=%u state=%u",
__func__, fscache_cookie_state(cookie), state);
fscache_print_cookie(cookie, 'O');
once_only = true;
}
goto again;
not_live:
spin_unlock(&cookie->lock);
failed:
cres->cache_priv = NULL;
cres->ops = NULL;
fscache_end_cookie_access(cookie, fscache_access_io_not_live);
_leave(" = -ENOBUFS");
return -ENOBUFS;
}
int __fscache_begin_read_operation(struct netfs_cache_resources *cres,
struct fscache_cookie *cookie)
{
return fscache_begin_operation(cres, cookie, FSCACHE_WANT_PARAMS,
fscache_access_io_read);
}
EXPORT_SYMBOL(__fscache_begin_read_operation);
int __fscache_begin_write_operation(struct netfs_cache_resources *cres,
struct fscache_cookie *cookie)
{
return fscache_begin_operation(cres, cookie, FSCACHE_WANT_PARAMS,
fscache_access_io_write);
}
EXPORT_SYMBOL(__fscache_begin_write_operation);
/**
* fscache_set_page_dirty - Mark page dirty and pin a cache object for writeback
* @page: The page being dirtied
* @cookie: The cookie referring to the cache object
*
* Set the dirty flag on a page and pin an in-use cache object in memory when
* dirtying a page so that writeback can later write to it. This is intended
* to be called from the filesystem's ->set_page_dirty() method.
*
* Returns 1 if PG_dirty was set on the page, 0 otherwise.
*/
int fscache_set_page_dirty(struct page *page, struct fscache_cookie *cookie)
{
struct inode *inode = page->mapping->host;
bool need_use = false;
_enter("");
if (!__set_page_dirty_nobuffers(page))
return 0;
if (!fscache_cookie_valid(cookie))
return 1;
if (!(inode->i_state & I_PINNING_FSCACHE_WB)) {
spin_lock(&inode->i_lock);
if (!(inode->i_state & I_PINNING_FSCACHE_WB)) {
inode->i_state |= I_PINNING_FSCACHE_WB;
need_use = true;
}
spin_unlock(&inode->i_lock);
if (need_use)
fscache_use_cookie(cookie, true);
}
return 1;
}
EXPORT_SYMBOL(fscache_set_page_dirty);
struct fscache_write_request {
struct netfs_cache_resources cache_resources;
struct address_space *mapping;
loff_t start;
size_t len;
bool set_bits;
netfs_io_terminated_t term_func;
void *term_func_priv;
};
void __fscache_clear_page_bits(struct address_space *mapping,
loff_t start, size_t len)
{
pgoff_t first = start / PAGE_SIZE;
pgoff_t last = (start + len - 1) / PAGE_SIZE;
struct page *page;
if (len) {
XA_STATE(xas, &mapping->i_pages, first);
rcu_read_lock();
xas_for_each(&xas, page, last) {
end_page_fscache(page);
}
rcu_read_unlock();
}
}
EXPORT_SYMBOL(__fscache_clear_page_bits);
/*
* Deal with the completion of writing the data to the cache.
*/
static void fscache_wreq_done(void *priv, ssize_t transferred_or_error,
bool was_async)
{
struct fscache_write_request *wreq = priv;
fscache_clear_page_bits(fscache_cres_cookie(&wreq->cache_resources),
wreq->mapping, wreq->start, wreq->len,
wreq->set_bits);
if (wreq->term_func)
wreq->term_func(wreq->term_func_priv, transferred_or_error,
was_async);
fscache_end_operation(&wreq->cache_resources);
kfree(wreq);
}
void __fscache_write_to_cache(struct fscache_cookie *cookie,
struct address_space *mapping,
loff_t start, size_t len, loff_t i_size,
netfs_io_terminated_t term_func,
void *term_func_priv,
bool cond)
{
struct fscache_write_request *wreq;
struct netfs_cache_resources *cres;
struct iov_iter iter;
int ret = -ENOBUFS;
if (len == 0)
goto abandon;
_enter("%llx,%zx", start, len);
wreq = kzalloc(sizeof(struct fscache_write_request), GFP_NOFS);
if (!wreq)
goto abandon;
wreq->mapping = mapping;
wreq->start = start;
wreq->len = len;
wreq->set_bits = cond;
wreq->term_func = term_func;
wreq->term_func_priv = term_func_priv;
cres = &wreq->cache_resources;
if (fscache_begin_operation(cres, cookie, FSCACHE_WANT_WRITE,
fscache_access_io_write) < 0)
goto abandon_free;
ret = cres->ops->prepare_write(cres, &start, &len, i_size, false);
if (ret < 0)
goto abandon_end;
/* TODO: Consider clearing page bits now for space the write isn't
* covering. This is more complicated than it appears when THPs are
* taken into account.
*/
iov_iter_xarray(&iter, WRITE, &mapping->i_pages, start, len);
fscache_write(cres, start, &iter, fscache_wreq_done, wreq);
return;
abandon_end:
return fscache_wreq_done(wreq, ret, false);
abandon_free:
kfree(wreq);
abandon:
fscache_clear_page_bits(cookie, mapping, start, len, cond);
if (term_func)
term_func(term_func_priv, ret, false);
}
EXPORT_SYMBOL(__fscache_write_to_cache);
/*
* Change the size of a backing object.
*/
void __fscache_resize_cookie(struct fscache_cookie *cookie, loff_t new_size)
{
struct netfs_cache_resources cres;
trace_fscache_resize(cookie, new_size);
if (fscache_begin_operation(&cres, cookie, FSCACHE_WANT_WRITE,
fscache_access_io_resize) == 0) {
fscache_stat(&fscache_n_resizes);
set_bit(FSCACHE_COOKIE_NEEDS_UPDATE, &cookie->flags);
/* We cannot defer a resize as we need to do it inside the
* netfs's inode lock so that we're serialised with respect to
* writes.
*/
cookie->volume->cache->ops->resize_cookie(&cres, new_size);
fscache_end_operation(&cres);
} else {
fscache_stat(&fscache_n_resizes_null);
}
}
EXPORT_SYMBOL(__fscache_resize_cookie);

147
fs/fscache/main.c
View File

@ -1,17 +1,13 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* General filesystem local caching manager
*
* Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/completion.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
#define CREATE_TRACE_POINTS
#include "internal.h"
@ -19,79 +15,18 @@ MODULE_DESCRIPTION("FS Cache Manager");
MODULE_AUTHOR("Red Hat, Inc.");
MODULE_LICENSE("GPL");
unsigned fscache_defer_lookup = 1;
module_param_named(defer_lookup, fscache_defer_lookup, uint,
S_IWUSR | S_IRUGO);
MODULE_PARM_DESC(fscache_defer_lookup,
"Defer cookie lookup to background thread");
unsigned fscache_defer_create = 1;
module_param_named(defer_create, fscache_defer_create, uint,
S_IWUSR | S_IRUGO);
MODULE_PARM_DESC(fscache_defer_create,
"Defer cookie creation to background thread");
unsigned fscache_debug;
module_param_named(debug, fscache_debug, uint,
S_IWUSR | S_IRUGO);
MODULE_PARM_DESC(fscache_debug,
"FS-Cache debugging mask");
struct kobject *fscache_root;
struct workqueue_struct *fscache_object_wq;
struct workqueue_struct *fscache_op_wq;
EXPORT_TRACEPOINT_SYMBOL(fscache_access_cache);
EXPORT_TRACEPOINT_SYMBOL(fscache_access_volume);
EXPORT_TRACEPOINT_SYMBOL(fscache_access);
DEFINE_PER_CPU(wait_queue_head_t, fscache_object_cong_wait);
/* these values serve as lower bounds, will be adjusted in fscache_init() */
static unsigned fscache_object_max_active = 4;
static unsigned fscache_op_max_active = 2;
#ifdef CONFIG_SYSCTL
static struct ctl_table_header *fscache_sysctl_header;
static int fscache_max_active_sysctl(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos)
{
struct workqueue_struct **wqp = table->extra1;
unsigned int *datap = table->data;
int ret;
ret = proc_dointvec(table, write, buffer, lenp, ppos);
if (ret == 0)
workqueue_set_max_active(*wqp, *datap);
return ret;
}
static struct ctl_table fscache_sysctls[] = {
{
.procname = "object_max_active",
.data = &fscache_object_max_active,
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = fscache_max_active_sysctl,
.extra1 = &fscache_object_wq,
},
{
.procname = "operation_max_active",
.data = &fscache_op_max_active,
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = fscache_max_active_sysctl,
.extra1 = &fscache_op_wq,
},
{}
};
static struct ctl_table fscache_sysctls_root[] = {
{
.procname = "fscache",
.mode = 0555,
.child = fscache_sysctls,
},
{}
};
#endif
struct workqueue_struct *fscache_wq;
EXPORT_SYMBOL(fscache_wq);
/*
* Mixing scores (in bits) for (7,20):
@ -118,15 +53,16 @@ static inline unsigned int fold_hash(unsigned long x, unsigned long y)
/*
* Generate a hash. This is derived from full_name_hash(), but we want to be
* sure it is arch independent and that it doesn't change as bits of the
* computed hash value might appear on disk. The caller also guarantees that
* the hashed data will be a series of aligned 32-bit words.
* computed hash value might appear on disk. The caller must guarantee that
* the source data is a multiple of four bytes in size.
*/
unsigned int fscache_hash(unsigned int salt, unsigned int *data, unsigned int n)
unsigned int fscache_hash(unsigned int salt, const void *data, size_t len)
{
unsigned int a, x = 0, y = salt;
const __le32 *p = data;
unsigned int a, x = 0, y = salt, n = len / sizeof(__le32);
for (; n; n--) {
a = *data++;
a = le32_to_cpu(*p++);
HASH_MIX(x, y, a);
}
return fold_hash(x, y);
@ -137,44 +73,16 @@ unsigned int fscache_hash(unsigned int salt, unsigned int *data, unsigned int n)
*/
static int __init fscache_init(void)
{
unsigned int nr_cpus = num_possible_cpus();
unsigned int cpu;
int ret;
int ret = -ENOMEM;
fscache_object_max_active =
clamp_val(nr_cpus,
fscache_object_max_active, WQ_UNBOUND_MAX_ACTIVE);
ret = -ENOMEM;
fscache_object_wq = alloc_workqueue("fscache_object", WQ_UNBOUND,
fscache_object_max_active);
if (!fscache_object_wq)
goto error_object_wq;
fscache_op_max_active =
clamp_val(fscache_object_max_active / 2,
fscache_op_max_active, WQ_UNBOUND_MAX_ACTIVE);
ret = -ENOMEM;
fscache_op_wq = alloc_workqueue("fscache_operation", WQ_UNBOUND,
fscache_op_max_active);
if (!fscache_op_wq)
goto error_op_wq;
for_each_possible_cpu(cpu)
init_waitqueue_head(&per_cpu(fscache_object_cong_wait, cpu));
fscache_wq = alloc_workqueue("fscache", WQ_UNBOUND | WQ_FREEZABLE, 0);
if (!fscache_wq)
goto error_wq;
ret = fscache_proc_init();
if (ret < 0)
goto error_proc;
#ifdef CONFIG_SYSCTL
ret = -ENOMEM;
fscache_sysctl_header = register_sysctl_table(fscache_sysctls_root);
if (!fscache_sysctl_header)
goto error_sysctl;
#endif
fscache_cookie_jar = kmem_cache_create("fscache_cookie_jar",
sizeof(struct fscache_cookie),
0, 0, NULL);
@ -184,26 +92,14 @@ static int __init fscache_init(void)
goto error_cookie_jar;
}
fscache_root = kobject_create_and_add("fscache", kernel_kobj);
if (!fscache_root)
goto error_kobj;
pr_notice("Loaded\n");
return 0;
error_kobj:
kmem_cache_destroy(fscache_cookie_jar);
error_cookie_jar:
#ifdef CONFIG_SYSCTL
unregister_sysctl_table(fscache_sysctl_header);
error_sysctl:
#endif
fscache_proc_cleanup();
error_proc:
destroy_workqueue(fscache_op_wq);
error_op_wq:
destroy_workqueue(fscache_object_wq);
error_object_wq:
destroy_workqueue(fscache_wq);
error_wq:
return ret;
}
@ -216,14 +112,9 @@ static void __exit fscache_exit(void)
{
_enter("");
kobject_put(fscache_root);
kmem_cache_destroy(fscache_cookie_jar);
#ifdef CONFIG_SYSCTL
unregister_sysctl_table(fscache_sysctl_header);
#endif
fscache_proc_cleanup();
destroy_workqueue(fscache_op_wq);
destroy_workqueue(fscache_object_wq);
destroy_workqueue(fscache_wq);
pr_notice("Unloaded\n");
}

74
fs/fscache/netfs.c
View File

@ -1,74 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache netfs (client) registration
*
* Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL COOKIE
#include <linux/module.h>
#include <linux/slab.h>
#include "internal.h"
/*
* register a network filesystem for caching
*/
int __fscache_register_netfs(struct fscache_netfs *netfs)
{
struct fscache_cookie *candidate, *cookie;
_enter("{%s}", netfs->name);
/* allocate a cookie for the primary index */
candidate = fscache_alloc_cookie(&fscache_fsdef_index,
&fscache_fsdef_netfs_def,
netfs->name, strlen(netfs->name),
&netfs->version, sizeof(netfs->version),
netfs, 0);
if (!candidate) {
_leave(" = -ENOMEM");
return -ENOMEM;
}
candidate->flags = 1 << FSCACHE_COOKIE_ENABLED;
/* check the netfs type is not already present */
cookie = fscache_hash_cookie(candidate);
if (!cookie)
goto already_registered;
if (cookie != candidate) {
trace_fscache_cookie(candidate->debug_id, 1, fscache_cookie_discard);
fscache_free_cookie(candidate);
}
fscache_cookie_get(cookie->parent, fscache_cookie_get_register_netfs);
atomic_inc(&cookie->parent->n_children);
netfs->primary_index = cookie;
pr_notice("Netfs '%s' registered for caching\n", netfs->name);
trace_fscache_netfs(netfs);
_leave(" = 0");
return 0;
already_registered:
fscache_cookie_put(candidate, fscache_cookie_put_dup_netfs);
_leave(" = -EEXIST");
return -EEXIST;
}
EXPORT_SYMBOL(__fscache_register_netfs);
/*
* unregister a network filesystem from the cache
* - all cookies must have been released first
*/
void __fscache_unregister_netfs(struct fscache_netfs *netfs)
{
_enter("{%s.%u}", netfs->name, netfs->version);
fscache_relinquish_cookie(netfs->primary_index, NULL, false);
pr_notice("Netfs '%s' unregistered from caching\n", netfs->name);
_leave("");
}
EXPORT_SYMBOL(__fscache_unregister_netfs);

1125
fs/fscache/object.c
View File

File diff suppressed because it is too large Load Diff

633
fs/fscache/operation.c
View File

@ -1,633 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache worker operation management routines
*
* Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*
* See Documentation/filesystems/caching/operations.rst
*/
#define FSCACHE_DEBUG_LEVEL OPERATION
#include <linux/module.h>
#include <linux/seq_file.h>
#include <linux/slab.h>
#include "internal.h"
atomic_t fscache_op_debug_id;
EXPORT_SYMBOL(fscache_op_debug_id);
static void fscache_operation_dummy_cancel(struct fscache_operation *op)
{
}
/**
* fscache_operation_init - Do basic initialisation of an operation
* @cookie: The cookie to operate on
* @op: The operation to initialise
* @processor: The function to perform the operation
* @cancel: A function to handle operation cancellation
* @release: The release function to assign
*
* Do basic initialisation of an operation. The caller must still set flags,
* object and processor if needed.
*/
void fscache_operation_init(struct fscache_cookie *cookie,
struct fscache_operation *op,
fscache_operation_processor_t processor,
fscache_operation_cancel_t cancel,
fscache_operation_release_t release)
{
INIT_WORK(&op->work, fscache_op_work_func);
atomic_set(&op->usage, 1);
op->state = FSCACHE_OP_ST_INITIALISED;
op->debug_id = atomic_inc_return(&fscache_op_debug_id);
op->processor = processor;
op->cancel = cancel ?: fscache_operation_dummy_cancel;
op->release = release;
INIT_LIST_HEAD(&op->pend_link);
fscache_stat(&fscache_n_op_initialised);
trace_fscache_op(cookie, op, fscache_op_init);
}
EXPORT_SYMBOL(fscache_operation_init);
/**
* fscache_enqueue_operation - Enqueue an operation for processing
* @op: The operation to enqueue
*
* Enqueue an operation for processing by the FS-Cache thread pool.
*
* This will get its own ref on the object.
*/
void fscache_enqueue_operation(struct fscache_operation *op)
{
struct fscache_cookie *cookie = op->object->cookie;
_enter("{OBJ%x OP%x,%u}",
op->object->debug_id, op->debug_id, atomic_read(&op->usage));
ASSERT(list_empty(&op->pend_link));
ASSERT(op->processor != NULL);
ASSERT(fscache_object_is_available(op->object));
ASSERTCMP(atomic_read(&op->usage), >, 0);
ASSERTIFCMP(op->state != FSCACHE_OP_ST_IN_PROGRESS,
op->state, ==, FSCACHE_OP_ST_CANCELLED);
fscache_stat(&fscache_n_op_enqueue);
switch (op->flags & FSCACHE_OP_TYPE) {
case FSCACHE_OP_ASYNC:
trace_fscache_op(cookie, op, fscache_op_enqueue_async);
_debug("queue async");
atomic_inc(&op->usage);
if (!queue_work(fscache_op_wq, &op->work))
fscache_put_operation(op);
break;
case FSCACHE_OP_MYTHREAD:
trace_fscache_op(cookie, op, fscache_op_enqueue_mythread);
_debug("queue for caller's attention");
break;
default:
pr_err("Unexpected op type %lx", op->flags);
BUG();
break;
}
}
EXPORT_SYMBOL(fscache_enqueue_operation);
/*
* start an op running
*/
static void fscache_run_op(struct fscache_object *object,
struct fscache_operation *op)
{
ASSERTCMP(op->state, ==, FSCACHE_OP_ST_PENDING);
op->state = FSCACHE_OP_ST_IN_PROGRESS;
object->n_in_progress++;
if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
if (op->processor)
fscache_enqueue_operation(op);
else
trace_fscache_op(object->cookie, op, fscache_op_run);
fscache_stat(&fscache_n_op_run);
}
/*
* report an unexpected submission
*/
static void fscache_report_unexpected_submission(struct fscache_object *object,
struct fscache_operation *op,
const struct fscache_state *ostate)
{
static bool once_only;
struct fscache_operation *p;
unsigned n;
if (once_only)
return;
once_only = true;
kdebug("unexpected submission OP%x [OBJ%x %s]",
op->debug_id, object->debug_id, object->state->name);
kdebug("objstate=%s [%s]", object->state->name, ostate->name);
kdebug("objflags=%lx", object->flags);
kdebug("objevent=%lx [%lx]", object->events, object->event_mask);
kdebug("ops=%u inp=%u exc=%u",
object->n_ops, object->n_in_progress, object->n_exclusive);
if (!list_empty(&object->pending_ops)) {
n = 0;
list_for_each_entry(p, &object->pending_ops, pend_link) {
ASSERTCMP(p->object, ==, object);
kdebug("%p %p", op->processor, op->release);
n++;
}
kdebug("n=%u", n);
}
dump_stack();
}
/*
* submit an exclusive operation for an object
* - other ops are excluded from running simultaneously with this one
* - this gets any extra refs it needs on an op
*/
int fscache_submit_exclusive_op(struct fscache_object *object,
struct fscache_operation *op)
{
const struct fscache_state *ostate;
unsigned long flags;
int ret;
_enter("{OBJ%x OP%x},", object->debug_id, op->debug_id);
trace_fscache_op(object->cookie, op, fscache_op_submit_ex);
ASSERTCMP(op->state, ==, FSCACHE_OP_ST_INITIALISED);
ASSERTCMP(atomic_read(&op->usage), >, 0);
spin_lock(&object->lock);
ASSERTCMP(object->n_ops, >=, object->n_in_progress);
ASSERTCMP(object->n_ops, >=, object->n_exclusive);
ASSERT(list_empty(&op->pend_link));
ostate = object->state;
smp_rmb();
op->state = FSCACHE_OP_ST_PENDING;
flags = READ_ONCE(object->flags);
if (unlikely(!(flags & BIT(FSCACHE_OBJECT_IS_LIVE)))) {
fscache_stat(&fscache_n_op_rejected);
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -ENOBUFS;
} else if (unlikely(fscache_cache_is_broken(object))) {
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -EIO;
} else if (flags & BIT(FSCACHE_OBJECT_IS_AVAILABLE)) {
op->object = object;
object->n_ops++;
object->n_exclusive++; /* reads and writes must wait */
if (object->n_in_progress > 0) {
atomic_inc(&op->usage);
list_add_tail(&op->pend_link, &object->pending_ops);
fscache_stat(&fscache_n_op_pend);
} else if (!list_empty(&object->pending_ops)) {
atomic_inc(&op->usage);
list_add_tail(&op->pend_link, &object->pending_ops);
fscache_stat(&fscache_n_op_pend);
fscache_start_operations(object);
} else {
ASSERTCMP(object->n_in_progress, ==, 0);
fscache_run_op(object, op);
}
/* need to issue a new write op after this */
clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
ret = 0;
} else if (flags & BIT(FSCACHE_OBJECT_IS_LOOKED_UP)) {
op->object = object;
object->n_ops++;
object->n_exclusive++; /* reads and writes must wait */
atomic_inc(&op->usage);
list_add_tail(&op->pend_link, &object->pending_ops);
fscache_stat(&fscache_n_op_pend);
ret = 0;
} else if (flags & BIT(FSCACHE_OBJECT_KILLED_BY_CACHE)) {
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -ENOBUFS;
} else {
fscache_report_unexpected_submission(object, op, ostate);
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -ENOBUFS;
}
spin_unlock(&object->lock);
return ret;
}
/*
* submit an operation for an object
* - objects may be submitted only in the following states:
* - during object creation (write ops may be submitted)
* - whilst the object is active
* - after an I/O error incurred in one of the two above states (op rejected)
* - this gets any extra refs it needs on an op
*/
int fscache_submit_op(struct fscache_object *object,
struct fscache_operation *op)
{
const struct fscache_state *ostate;
unsigned long flags;
int ret;
_enter("{OBJ%x OP%x},{%u}",
object->debug_id, op->debug_id, atomic_read(&op->usage));
trace_fscache_op(object->cookie, op, fscache_op_submit);
ASSERTCMP(op->state, ==, FSCACHE_OP_ST_INITIALISED);
ASSERTCMP(atomic_read(&op->usage), >, 0);
spin_lock(&object->lock);
ASSERTCMP(object->n_ops, >=, object->n_in_progress);
ASSERTCMP(object->n_ops, >=, object->n_exclusive);
ASSERT(list_empty(&op->pend_link));
ostate = object->state;
smp_rmb();
op->state = FSCACHE_OP_ST_PENDING;
flags = READ_ONCE(object->flags);
if (unlikely(!(flags & BIT(FSCACHE_OBJECT_IS_LIVE)))) {
fscache_stat(&fscache_n_op_rejected);
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -ENOBUFS;
} else if (unlikely(fscache_cache_is_broken(object))) {
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -EIO;
} else if (flags & BIT(FSCACHE_OBJECT_IS_AVAILABLE)) {
op->object = object;
object->n_ops++;
if (object->n_exclusive > 0) {
atomic_inc(&op->usage);
list_add_tail(&op->pend_link, &object->pending_ops);
fscache_stat(&fscache_n_op_pend);
} else if (!list_empty(&object->pending_ops)) {
atomic_inc(&op->usage);
list_add_tail(&op->pend_link, &object->pending_ops);
fscache_stat(&fscache_n_op_pend);
fscache_start_operations(object);
} else {
ASSERTCMP(object->n_exclusive, ==, 0);
fscache_run_op(object, op);
}
ret = 0;
} else if (flags & BIT(FSCACHE_OBJECT_IS_LOOKED_UP)) {
op->object = object;
object->n_ops++;
atomic_inc(&op->usage);
list_add_tail(&op->pend_link, &object->pending_ops);
fscache_stat(&fscache_n_op_pend);
ret = 0;
} else if (flags & BIT(FSCACHE_OBJECT_KILLED_BY_CACHE)) {
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -ENOBUFS;
} else {
fscache_report_unexpected_submission(object, op, ostate);
ASSERT(!fscache_object_is_active(object));
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
ret = -ENOBUFS;
}
spin_unlock(&object->lock);
return ret;
}
/*
* queue an object for withdrawal on error, aborting all following asynchronous
* operations
*/
void fscache_abort_object(struct fscache_object *object)
{
_enter("{OBJ%x}", object->debug_id);
fscache_raise_event(object, FSCACHE_OBJECT_EV_ERROR);
}
/*
* Jump start the operation processing on an object. The caller must hold
* object->lock.
*/
void fscache_start_operations(struct fscache_object *object)
{
struct fscache_operation *op;
bool stop = false;
while (!list_empty(&object->pending_ops) && !stop) {
op = list_entry(object->pending_ops.next,
struct fscache_operation, pend_link);
if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags)) {
if (object->n_in_progress > 0)
break;
stop = true;
}
list_del_init(&op->pend_link);
fscache_run_op(object, op);
/* the pending queue was holding a ref on the object */
fscache_put_operation(op);
}
ASSERTCMP(object->n_in_progress, <=, object->n_ops);
_debug("woke %d ops on OBJ%x",
object->n_in_progress, object->debug_id);
}
/*
* cancel an operation that's pending on an object
*/
int fscache_cancel_op(struct fscache_operation *op,
bool cancel_in_progress_op)
{
struct fscache_object *object = op->object;
bool put = false;
int ret;
_enter("OBJ%x OP%x}", op->object->debug_id, op->debug_id);
trace_fscache_op(object->cookie, op, fscache_op_cancel);
ASSERTCMP(op->state, >=, FSCACHE_OP_ST_PENDING);
ASSERTCMP(op->state, !=, FSCACHE_OP_ST_CANCELLED);
ASSERTCMP(atomic_read(&op->usage), >, 0);
spin_lock(&object->lock);
ret = -EBUSY;
if (op->state == FSCACHE_OP_ST_PENDING) {
ASSERT(!list_empty(&op->pend_link));
list_del_init(&op->pend_link);
put = true;
fscache_stat(&fscache_n_op_cancelled);
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
object->n_exclusive--;
if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
ret = 0;
} else if (op->state == FSCACHE_OP_ST_IN_PROGRESS && cancel_in_progress_op) {
ASSERTCMP(object->n_in_progress, >, 0);
if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
object->n_exclusive--;
object->n_in_progress--;
if (object->n_in_progress == 0)
fscache_start_operations(object);
fscache_stat(&fscache_n_op_cancelled);
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
object->n_exclusive--;
if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
ret = 0;
}
if (put)
fscache_put_operation(op);
spin_unlock(&object->lock);
_leave(" = %d", ret);
return ret;
}
/*
* Cancel all pending operations on an object
*/
void fscache_cancel_all_ops(struct fscache_object *object)
{
struct fscache_operation *op;
_enter("OBJ%x", object->debug_id);
spin_lock(&object->lock);
while (!list_empty(&object->pending_ops)) {
op = list_entry(object->pending_ops.next,
struct fscache_operation, pend_link);
fscache_stat(&fscache_n_op_cancelled);
list_del_init(&op->pend_link);
trace_fscache_op(object->cookie, op, fscache_op_cancel_all);
ASSERTCMP(op->state, ==, FSCACHE_OP_ST_PENDING);
op->cancel(op);
op->state = FSCACHE_OP_ST_CANCELLED;
if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
object->n_exclusive--;
if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
fscache_put_operation(op);
cond_resched_lock(&object->lock);
}
spin_unlock(&object->lock);
_leave("");
}
/*
* Record the completion or cancellation of an in-progress operation.
*/
void fscache_op_complete(struct fscache_operation *op, bool cancelled)
{
struct fscache_object *object = op->object;
_enter("OBJ%x", object->debug_id);
ASSERTCMP(op->state, ==, FSCACHE_OP_ST_IN_PROGRESS);
ASSERTCMP(object->n_in_progress, >, 0);
ASSERTIFCMP(test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags),
object->n_exclusive, >, 0);
ASSERTIFCMP(test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags),
object->n_in_progress, ==, 1);
spin_lock(&object->lock);
if (!cancelled) {
trace_fscache_op(object->cookie, op, fscache_op_completed);
op->state = FSCACHE_OP_ST_COMPLETE;
} else {
op->cancel(op);
trace_fscache_op(object->cookie, op, fscache_op_cancelled);
op->state = FSCACHE_OP_ST_CANCELLED;
}
if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
object->n_exclusive--;
object->n_in_progress--;
if (object->n_in_progress == 0)
fscache_start_operations(object);
spin_unlock(&object->lock);
_leave("");
}
EXPORT_SYMBOL(fscache_op_complete);
/*
* release an operation
* - queues pending ops if this is the last in-progress op
*/
void fscache_put_operation(struct fscache_operation *op)
{
struct fscache_object *object;
struct fscache_cache *cache;
_enter("{OBJ%x OP%x,%d}",
op->object ? op->object->debug_id : 0,
op->debug_id, atomic_read(&op->usage));
ASSERTCMP(atomic_read(&op->usage), >, 0);
if (!atomic_dec_and_test(&op->usage))
return;
trace_fscache_op(op->object ? op->object->cookie : NULL, op, fscache_op_put);
_debug("PUT OP");
ASSERTIFCMP(op->state != FSCACHE_OP_ST_INITIALISED &&
op->state != FSCACHE_OP_ST_COMPLETE,
op->state, ==, FSCACHE_OP_ST_CANCELLED);
fscache_stat(&fscache_n_op_release);
if (op->release) {
op->release(op);
op->release = NULL;
}
op->state = FSCACHE_OP_ST_DEAD;
object = op->object;
if (likely(object)) {
if (test_bit(FSCACHE_OP_DEC_READ_CNT, &op->flags))
atomic_dec(&object->n_reads);
if (test_bit(FSCACHE_OP_UNUSE_COOKIE, &op->flags))
fscache_unuse_cookie(object);
/* now... we may get called with the object spinlock held, so we
* complete the cleanup here only if we can immediately acquire the
* lock, and defer it otherwise */
if (!spin_trylock(&object->lock)) {
_debug("defer put");
fscache_stat(&fscache_n_op_deferred_release);
cache = object->cache;
spin_lock(&cache->op_gc_list_lock);
list_add_tail(&op->pend_link, &cache->op_gc_list);
spin_unlock(&cache->op_gc_list_lock);
schedule_work(&cache->op_gc);
_leave(" [defer]");
return;
}
ASSERTCMP(object->n_ops, >, 0);
object->n_ops--;
if (object->n_ops == 0)
fscache_raise_event(object, FSCACHE_OBJECT_EV_CLEARED);
spin_unlock(&object->lock);
}
kfree(op);
_leave(" [done]");
}
EXPORT_SYMBOL(fscache_put_operation);
/*
* garbage collect operations that have had their release deferred
*/
void fscache_operation_gc(struct work_struct *work)
{
struct fscache_operation *op;
struct fscache_object *object;
struct fscache_cache *cache =
container_of(work, struct fscache_cache, op_gc);
int count = 0;
_enter("");
do {
spin_lock(&cache->op_gc_list_lock);
if (list_empty(&cache->op_gc_list)) {
spin_unlock(&cache->op_gc_list_lock);
break;
}
op = list_entry(cache->op_gc_list.next,
struct fscache_operation, pend_link);
list_del(&op->pend_link);
spin_unlock(&cache->op_gc_list_lock);
object = op->object;
trace_fscache_op(object->cookie, op, fscache_op_gc);
spin_lock(&object->lock);
_debug("GC DEFERRED REL OBJ%x OP%x",
object->debug_id, op->debug_id);
fscache_stat(&fscache_n_op_gc);
ASSERTCMP(atomic_read(&op->usage), ==, 0);
ASSERTCMP(op->state, ==, FSCACHE_OP_ST_DEAD);
ASSERTCMP(object->n_ops, >, 0);
object->n_ops--;
if (object->n_ops == 0)
fscache_raise_event(object, FSCACHE_OBJECT_EV_CLEARED);
spin_unlock(&object->lock);
kfree(op);
} while (count++ < 20);
if (!list_empty(&cache->op_gc_list))
schedule_work(&cache->op_gc);
_leave("");
}
/*
* execute an operation using fs_op_wq to provide processing context -
* the caller holds a ref to this object, so we don't need to hold one
*/
void fscache_op_work_func(struct work_struct *work)
{
struct fscache_operation *op =
container_of(work, struct fscache_operation, work);
_enter("{OBJ%x OP%x,%d}",
op->object->debug_id, op->debug_id, atomic_read(&op->usage));
trace_fscache_op(op->object->cookie, op, fscache_op_work);
ASSERT(op->processor != NULL);
op->processor(op);
fscache_put_operation(op);
_leave("");
}

1242
fs/fscache/page.c
View File

File diff suppressed because it is too large Load Diff

47
fs/fscache/proc.c
View File

@ -1,11 +1,11 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache statistics viewing interface
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL OPERATION
#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
@ -16,42 +16,32 @@
*/
int __init fscache_proc_init(void)
{
_enter("");
if (!proc_mkdir("fs/fscache", NULL))
goto error_dir;
if (!proc_create_seq("fs/fscache/caches", S_IFREG | 0444, NULL,
&fscache_caches_seq_ops))
goto error;
if (!proc_create_seq("fs/fscache/volumes", S_IFREG | 0444, NULL,
&fscache_volumes_seq_ops))
goto error;
if (!proc_create_seq("fs/fscache/cookies", S_IFREG | 0444, NULL,
&fscache_cookies_seq_ops))
goto error_cookies;
goto error;
#ifdef CONFIG_FSCACHE_STATS
if (!proc_create_single("fs/fscache/stats", S_IFREG | 0444, NULL,
fscache_stats_show))
goto error_stats;
fscache_stats_show))
goto error;
#endif
#ifdef CONFIG_FSCACHE_OBJECT_LIST
if (!proc_create("fs/fscache/objects", S_IFREG | 0444, NULL,
&fscache_objlist_proc_ops))
goto error_objects;
#endif
_leave(" = 0");
return 0;
#ifdef CONFIG_FSCACHE_OBJECT_LIST
error_objects:
#endif
#ifdef CONFIG_FSCACHE_STATS
remove_proc_entry("fs/fscache/stats", NULL);
error_stats:
#endif
remove_proc_entry("fs/fscache/cookies", NULL);
error_cookies:
error:
remove_proc_entry("fs/fscache", NULL);
error_dir:
_leave(" = -ENOMEM");
return -ENOMEM;
}
@ -60,12 +50,5 @@ int __init fscache_proc_init(void)
*/
void fscache_proc_cleanup(void)
{
#ifdef CONFIG_FSCACHE_OBJECT_LIST
remove_proc_entry("fs/fscache/objects", NULL);
#endif
#ifdef CONFIG_FSCACHE_STATS
remove_proc_entry("fs/fscache/stats", NULL);
#endif
remove_proc_entry("fs/fscache/cookies", NULL);
remove_proc_entry("fs/fscache", NULL);
remove_proc_subtree("fs/fscache", NULL);
}

287
fs/fscache/stats.c
View File

@ -1,12 +1,11 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache statistics
*
* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL THREAD
#include <linux/module.h>
#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include "internal.h"
@ -14,122 +13,41 @@
/*
* operation counters
*/
atomic_t fscache_n_op_pend;
atomic_t fscache_n_op_run;
atomic_t fscache_n_op_enqueue;
atomic_t fscache_n_op_deferred_release;
atomic_t fscache_n_op_initialised;
atomic_t fscache_n_op_release;
atomic_t fscache_n_op_gc;
atomic_t fscache_n_op_cancelled;
atomic_t fscache_n_op_rejected;
atomic_t fscache_n_attr_changed;
atomic_t fscache_n_attr_changed_ok;
atomic_t fscache_n_attr_changed_nobufs;
atomic_t fscache_n_attr_changed_nomem;
atomic_t fscache_n_attr_changed_calls;
atomic_t fscache_n_allocs;
atomic_t fscache_n_allocs_ok;
atomic_t fscache_n_allocs_wait;
atomic_t fscache_n_allocs_nobufs;
atomic_t fscache_n_allocs_intr;
atomic_t fscache_n_allocs_object_dead;
atomic_t fscache_n_alloc_ops;
atomic_t fscache_n_alloc_op_waits;
atomic_t fscache_n_retrievals;
atomic_t fscache_n_retrievals_ok;
atomic_t fscache_n_retrievals_wait;
atomic_t fscache_n_retrievals_nodata;
atomic_t fscache_n_retrievals_nobufs;
atomic_t fscache_n_retrievals_intr;
atomic_t fscache_n_retrievals_nomem;
atomic_t fscache_n_retrievals_object_dead;
atomic_t fscache_n_retrieval_ops;
atomic_t fscache_n_retrieval_op_waits;
atomic_t fscache_n_stores;
atomic_t fscache_n_stores_ok;
atomic_t fscache_n_stores_again;
atomic_t fscache_n_stores_nobufs;
atomic_t fscache_n_stores_oom;
atomic_t fscache_n_store_ops;
atomic_t fscache_n_store_calls;
atomic_t fscache_n_store_pages;
atomic_t fscache_n_store_radix_deletes;
atomic_t fscache_n_store_pages_over_limit;
atomic_t fscache_n_store_vmscan_not_storing;
atomic_t fscache_n_store_vmscan_gone;
atomic_t fscache_n_store_vmscan_busy;
atomic_t fscache_n_store_vmscan_cancelled;
atomic_t fscache_n_store_vmscan_wait;
atomic_t fscache_n_marks;
atomic_t fscache_n_uncaches;
atomic_t fscache_n_volumes;
atomic_t fscache_n_volumes_collision;
atomic_t fscache_n_volumes_nomem;
atomic_t fscache_n_cookies;
atomic_t fscache_n_cookies_lru;
atomic_t fscache_n_cookies_lru_expired;
atomic_t fscache_n_cookies_lru_removed;
atomic_t fscache_n_cookies_lru_dropped;
atomic_t fscache_n_acquires;
atomic_t fscache_n_acquires_null;
atomic_t fscache_n_acquires_no_cache;
atomic_t fscache_n_acquires_ok;
atomic_t fscache_n_acquires_nobufs;
atomic_t fscache_n_acquires_oom;
atomic_t fscache_n_invalidates;
atomic_t fscache_n_invalidates_run;
atomic_t fscache_n_updates;
atomic_t fscache_n_updates_null;
atomic_t fscache_n_updates_run;
EXPORT_SYMBOL(fscache_n_updates);
atomic_t fscache_n_relinquishes;
atomic_t fscache_n_relinquishes_null;
atomic_t fscache_n_relinquishes_waitcrt;
atomic_t fscache_n_relinquishes_retire;
atomic_t fscache_n_relinquishes_dropped;
atomic_t fscache_n_cookie_index;
atomic_t fscache_n_cookie_data;
atomic_t fscache_n_cookie_special;
atomic_t fscache_n_resizes;
atomic_t fscache_n_resizes_null;
atomic_t fscache_n_object_alloc;
atomic_t fscache_n_object_no_alloc;
atomic_t fscache_n_object_lookups;
atomic_t fscache_n_object_lookups_negative;
atomic_t fscache_n_object_lookups_positive;
atomic_t fscache_n_object_lookups_timed_out;
atomic_t fscache_n_object_created;
atomic_t fscache_n_object_avail;
atomic_t fscache_n_object_dead;
atomic_t fscache_n_checkaux_none;
atomic_t fscache_n_checkaux_okay;
atomic_t fscache_n_checkaux_update;
atomic_t fscache_n_checkaux_obsolete;
atomic_t fscache_n_cop_alloc_object;
atomic_t fscache_n_cop_lookup_object;
atomic_t fscache_n_cop_lookup_complete;
atomic_t fscache_n_cop_grab_object;
atomic_t fscache_n_cop_invalidate_object;
atomic_t fscache_n_cop_update_object;
atomic_t fscache_n_cop_drop_object;
atomic_t fscache_n_cop_put_object;
atomic_t fscache_n_cop_sync_cache;
atomic_t fscache_n_cop_attr_changed;
atomic_t fscache_n_cop_read_or_alloc_page;
atomic_t fscache_n_cop_read_or_alloc_pages;
atomic_t fscache_n_cop_allocate_page;
atomic_t fscache_n_cop_allocate_pages;
atomic_t fscache_n_cop_write_page;
atomic_t fscache_n_cop_uncache_page;
atomic_t fscache_n_cop_dissociate_pages;
atomic_t fscache_n_cache_no_space_reject;
atomic_t fscache_n_cache_stale_objects;
atomic_t fscache_n_cache_retired_objects;
atomic_t fscache_n_cache_culled_objects;
atomic_t fscache_n_read;
EXPORT_SYMBOL(fscache_n_read);
atomic_t fscache_n_write;
EXPORT_SYMBOL(fscache_n_write);
atomic_t fscache_n_no_write_space;
EXPORT_SYMBOL(fscache_n_no_write_space);
atomic_t fscache_n_no_create_space;
EXPORT_SYMBOL(fscache_n_no_create_space);
atomic_t fscache_n_culled;
EXPORT_SYMBOL(fscache_n_culled);
/*
* display the general statistics
@ -137,147 +55,48 @@ atomic_t fscache_n_cache_culled_objects;
int fscache_stats_show(struct seq_file *m, void *v)
{
seq_puts(m, "FS-Cache statistics\n");
seq_printf(m, "Cookies: n=%d v=%d vcol=%u voom=%u\n",
atomic_read(&fscache_n_cookies),
atomic_read(&fscache_n_volumes),
atomic_read(&fscache_n_volumes_collision),
atomic_read(&fscache_n_volumes_nomem)
);
seq_printf(m, "Cookies: idx=%u dat=%u spc=%u\n",
atomic_read(&fscache_n_cookie_index),
atomic_read(&fscache_n_cookie_data),
atomic_read(&fscache_n_cookie_special));
seq_printf(m, "Objects: alc=%u nal=%u avl=%u ded=%u\n",
atomic_read(&fscache_n_object_alloc),
atomic_read(&fscache_n_object_no_alloc),
atomic_read(&fscache_n_object_avail),
atomic_read(&fscache_n_object_dead));
seq_printf(m, "ChkAux : non=%u ok=%u upd=%u obs=%u\n",
atomic_read(&fscache_n_checkaux_none),
atomic_read(&fscache_n_checkaux_okay),
atomic_read(&fscache_n_checkaux_update),
atomic_read(&fscache_n_checkaux_obsolete));
seq_printf(m, "Pages : mrk=%u unc=%u\n",
atomic_read(&fscache_n_marks),
atomic_read(&fscache_n_uncaches));
seq_printf(m, "Acquire: n=%u nul=%u noc=%u ok=%u nbf=%u"
" oom=%u\n",
seq_printf(m, "Acquire: n=%u ok=%u oom=%u\n",
atomic_read(&fscache_n_acquires),
atomic_read(&fscache_n_acquires_null),
atomic_read(&fscache_n_acquires_no_cache),
atomic_read(&fscache_n_acquires_ok),
atomic_read(&fscache_n_acquires_nobufs),
atomic_read(&fscache_n_acquires_oom));
seq_printf(m, "Lookups: n=%u neg=%u pos=%u crt=%u tmo=%u\n",
atomic_read(&fscache_n_object_lookups),
atomic_read(&fscache_n_object_lookups_negative),
atomic_read(&fscache_n_object_lookups_positive),
atomic_read(&fscache_n_object_created),
atomic_read(&fscache_n_object_lookups_timed_out));
seq_printf(m, "LRU : n=%u exp=%u rmv=%u drp=%u at=%ld\n",
atomic_read(&fscache_n_cookies_lru),
atomic_read(&fscache_n_cookies_lru_expired),
atomic_read(&fscache_n_cookies_lru_removed),
atomic_read(&fscache_n_cookies_lru_dropped),
timer_pending(&fscache_cookie_lru_timer) ?
fscache_cookie_lru_timer.expires - jiffies : 0);
seq_printf(m, "Invals : n=%u run=%u\n",
atomic_read(&fscache_n_invalidates),
atomic_read(&fscache_n_invalidates_run));
seq_printf(m, "Invals : n=%u\n",
atomic_read(&fscache_n_invalidates));
seq_printf(m, "Updates: n=%u nul=%u run=%u\n",
seq_printf(m, "Updates: n=%u rsz=%u rsn=%u\n",
atomic_read(&fscache_n_updates),
atomic_read(&fscache_n_updates_null),
atomic_read(&fscache_n_updates_run));
atomic_read(&fscache_n_resizes),
atomic_read(&fscache_n_resizes_null));
seq_printf(m, "Relinqs: n=%u nul=%u wcr=%u rtr=%u\n",
seq_printf(m, "Relinqs: n=%u rtr=%u drop=%u\n",
atomic_read(&fscache_n_relinquishes),
atomic_read(&fscache_n_relinquishes_null),
atomic_read(&fscache_n_relinquishes_waitcrt),
atomic_read(&fscache_n_relinquishes_retire));
atomic_read(&fscache_n_relinquishes_retire),
atomic_read(&fscache_n_relinquishes_dropped));
seq_printf(m, "AttrChg: n=%u ok=%u nbf=%u oom=%u run=%u\n",
atomic_read(&fscache_n_attr_changed),
atomic_read(&fscache_n_attr_changed_ok),
atomic_read(&fscache_n_attr_changed_nobufs),
atomic_read(&fscache_n_attr_changed_nomem),
atomic_read(&fscache_n_attr_changed_calls));
seq_printf(m, "NoSpace: nwr=%u ncr=%u cull=%u\n",
atomic_read(&fscache_n_no_write_space),
atomic_read(&fscache_n_no_create_space),
atomic_read(&fscache_n_culled));
seq_printf(m, "Allocs : n=%u ok=%u wt=%u nbf=%u int=%u\n",
atomic_read(&fscache_n_allocs),
atomic_read(&fscache_n_allocs_ok),
atomic_read(&fscache_n_allocs_wait),
atomic_read(&fscache_n_allocs_nobufs),
atomic_read(&fscache_n_allocs_intr));
seq_printf(m, "Allocs : ops=%u owt=%u abt=%u\n",
atomic_read(&fscache_n_alloc_ops),
atomic_read(&fscache_n_alloc_op_waits),
atomic_read(&fscache_n_allocs_object_dead));
seq_printf(m, "IO : rd=%u wr=%u\n",
atomic_read(&fscache_n_read),
atomic_read(&fscache_n_write));
seq_printf(m, "Retrvls: n=%u ok=%u wt=%u nod=%u nbf=%u"
" int=%u oom=%u\n",
atomic_read(&fscache_n_retrievals),
atomic_read(&fscache_n_retrievals_ok),
atomic_read(&fscache_n_retrievals_wait),
atomic_read(&fscache_n_retrievals_nodata),
atomic_read(&fscache_n_retrievals_nobufs),
atomic_read(&fscache_n_retrievals_intr),
atomic_read(&fscache_n_retrievals_nomem));
seq_printf(m, "Retrvls: ops=%u owt=%u abt=%u\n",
atomic_read(&fscache_n_retrieval_ops),
atomic_read(&fscache_n_retrieval_op_waits),
atomic_read(&fscache_n_retrievals_object_dead));
seq_printf(m, "Stores : n=%u ok=%u agn=%u nbf=%u oom=%u\n",
atomic_read(&fscache_n_stores),
atomic_read(&fscache_n_stores_ok),
atomic_read(&fscache_n_stores_again),
atomic_read(&fscache_n_stores_nobufs),
atomic_read(&fscache_n_stores_oom));
seq_printf(m, "Stores : ops=%u run=%u pgs=%u rxd=%u olm=%u\n",
atomic_read(&fscache_n_store_ops),
atomic_read(&fscache_n_store_calls),
atomic_read(&fscache_n_store_pages),
atomic_read(&fscache_n_store_radix_deletes),
atomic_read(&fscache_n_store_pages_over_limit));
seq_printf(m, "VmScan : nos=%u gon=%u bsy=%u can=%u wt=%u\n",
atomic_read(&fscache_n_store_vmscan_not_storing),
atomic_read(&fscache_n_store_vmscan_gone),
atomic_read(&fscache_n_store_vmscan_busy),
atomic_read(&fscache_n_store_vmscan_cancelled),
atomic_read(&fscache_n_store_vmscan_wait));
seq_printf(m, "Ops : pend=%u run=%u enq=%u can=%u rej=%u\n",
atomic_read(&fscache_n_op_pend),
atomic_read(&fscache_n_op_run),
atomic_read(&fscache_n_op_enqueue),
atomic_read(&fscache_n_op_cancelled),
atomic_read(&fscache_n_op_rejected));
seq_printf(m, "Ops : ini=%u dfr=%u rel=%u gc=%u\n",
atomic_read(&fscache_n_op_initialised),
atomic_read(&fscache_n_op_deferred_release),
atomic_read(&fscache_n_op_release),
atomic_read(&fscache_n_op_gc));
seq_printf(m, "CacheOp: alo=%d luo=%d luc=%d gro=%d\n",
atomic_read(&fscache_n_cop_alloc_object),
atomic_read(&fscache_n_cop_lookup_object),
atomic_read(&fscache_n_cop_lookup_complete),
atomic_read(&fscache_n_cop_grab_object));
seq_printf(m, "CacheOp: inv=%d upo=%d dro=%d pto=%d atc=%d syn=%d\n",
atomic_read(&fscache_n_cop_invalidate_object),
atomic_read(&fscache_n_cop_update_object),
atomic_read(&fscache_n_cop_drop_object),
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
atomic_read(&fscache_n_cop_sync_cache));
seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d dsp=%d\n",
atomic_read(&fscache_n_cop_read_or_alloc_page),
atomic_read(&fscache_n_cop_read_or_alloc_pages),
atomic_read(&fscache_n_cop_allocate_page),
atomic_read(&fscache_n_cop_allocate_pages),
atomic_read(&fscache_n_cop_write_page),
atomic_read(&fscache_n_cop_uncache_page),
atomic_read(&fscache_n_cop_dissociate_pages));
seq_printf(m, "CacheEv: nsp=%d stl=%d rtr=%d cul=%d\n",
atomic_read(&fscache_n_cache_no_space_reject),
atomic_read(&fscache_n_cache_stale_objects),
atomic_read(&fscache_n_cache_retired_objects),
atomic_read(&fscache_n_cache_culled_objects));
netfs_stats_show(m);
return 0;
}

517
fs/fscache/volume.c Normal file
View File

@ -0,0 +1,517 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* Volume-level cache cookie handling.
*
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#define FSCACHE_DEBUG_LEVEL COOKIE
#include <linux/export.h>
#include <linux/slab.h>
#include "internal.h"
#define fscache_volume_hash_shift 10
static struct hlist_bl_head fscache_volume_hash[1 << fscache_volume_hash_shift];
static atomic_t fscache_volume_debug_id;
static LIST_HEAD(fscache_volumes);
static void fscache_create_volume_work(struct work_struct *work);
struct fscache_volume *fscache_get_volume(struct fscache_volume *volume,
enum fscache_volume_trace where)
{
int ref;
__refcount_inc(&volume->ref, &ref);
trace_fscache_volume(volume->debug_id, ref + 1, where);
return volume;
}
static void fscache_see_volume(struct fscache_volume *volume,
enum fscache_volume_trace where)
{
int ref = refcount_read(&volume->ref);
trace_fscache_volume(volume->debug_id, ref, where);
}
/*
* Pin the cache behind a volume so that we can access it.
*/
static void __fscache_begin_volume_access(struct fscache_volume *volume,
struct fscache_cookie *cookie,
enum fscache_access_trace why)
{
int n_accesses;
n_accesses = atomic_inc_return(&volume->n_accesses);
smp_mb__after_atomic();
trace_fscache_access_volume(volume->debug_id, cookie ? cookie->debug_id : 0,
refcount_read(&volume->ref),
n_accesses, why);
}
/**
* fscache_begin_volume_access - Pin a cache so a volume can be accessed
* @volume: The volume cookie
* @cookie: A datafile cookie for a tracing reference (or NULL)
* @why: An indication of the circumstances of the access for tracing
*
* Attempt to pin the cache to prevent it from going away whilst we're
* accessing a volume and returns true if successful. This works as follows:
*
* (1) If the cache tests as not live (state is not FSCACHE_CACHE_IS_ACTIVE),
* then we return false to indicate access was not permitted.
*
* (2) If the cache tests as live, then we increment the volume's n_accesses
* count and then recheck the cache liveness, ending the access if it
* ceased to be live.
*
* (3) When we end the access, we decrement the volume's n_accesses and wake
* up the any waiters if it reaches 0.
*
* (4) Whilst the cache is caching, the volume's n_accesses is kept
* artificially incremented to prevent wakeups from happening.
*
* (5) When the cache is taken offline, the state is changed to prevent new
* accesses, the volume's n_accesses is decremented and we wait for it to
* become 0.
*
* The datafile @cookie and the @why indicator are merely provided for tracing
* purposes.
*/
bool fscache_begin_volume_access(struct fscache_volume *volume,
struct fscache_cookie *cookie,
enum fscache_access_trace why)
{
if (!fscache_cache_is_live(volume->cache))
return false;
__fscache_begin_volume_access(volume, cookie, why);
if (!fscache_cache_is_live(volume->cache)) {
fscache_end_volume_access(volume, cookie, fscache_access_unlive);
return false;
}
return true;
}
/**
* fscache_end_volume_access - Unpin a cache at the end of an access.
* @volume: The volume cookie
* @cookie: A datafile cookie for a tracing reference (or NULL)
* @why: An indication of the circumstances of the access for tracing
*
* Unpin a cache volume after we've accessed it. The datafile @cookie and the
* @why indicator are merely provided for tracing purposes.
*/
void fscache_end_volume_access(struct fscache_volume *volume,
struct fscache_cookie *cookie,
enum fscache_access_trace why)
{
int n_accesses;
smp_mb__before_atomic();
n_accesses = atomic_dec_return(&volume->n_accesses);
trace_fscache_access_volume(volume->debug_id, cookie ? cookie->debug_id : 0,
refcount_read(&volume->ref),
n_accesses, why);
if (n_accesses == 0)
wake_up_var(&volume->n_accesses);
}
EXPORT_SYMBOL(fscache_end_volume_access);
static bool fscache_volume_same(const struct fscache_volume *a,
const struct fscache_volume *b)
{
size_t klen;
if (a->key_hash != b->key_hash ||
a->cache != b->cache ||
a->key[0] != b->key[0])
return false;
klen = round_up(a->key[0] + 1, sizeof(__le32));
return memcmp(a->key, b->key, klen) == 0;
}
static bool fscache_is_acquire_pending(struct fscache_volume *volume)
{
return test_bit(FSCACHE_VOLUME_ACQUIRE_PENDING, &volume->flags);
}
static void fscache_wait_on_volume_collision(struct fscache_volume *candidate,
unsigned int collidee_debug_id)
{
wait_var_event_timeout(&candidate->flags,
fscache_is_acquire_pending(candidate), 20 * HZ);
if (!fscache_is_acquire_pending(candidate)) {
pr_notice("Potential volume collision new=%08x old=%08x",
candidate->debug_id, collidee_debug_id);
fscache_stat(&fscache_n_volumes_collision);
wait_var_event(&candidate->flags, fscache_is_acquire_pending(candidate));
}
}
/*
* Attempt to insert the new volume into the hash. If there's a collision, we
* wait for the old volume to complete if it's being relinquished and an error
* otherwise.
*/
static bool fscache_hash_volume(struct fscache_volume *candidate)
{
struct fscache_volume *cursor;
struct hlist_bl_head *h;
struct hlist_bl_node *p;
unsigned int bucket, collidee_debug_id = 0;
bucket = candidate->key_hash & (ARRAY_SIZE(fscache_volume_hash) - 1);
h = &fscache_volume_hash[bucket];
hlist_bl_lock(h);
hlist_bl_for_each_entry(cursor, p, h, hash_link) {
if (fscache_volume_same(candidate, cursor)) {
if (!test_bit(FSCACHE_VOLUME_RELINQUISHED, &cursor->flags))
goto collision;
fscache_see_volume(cursor, fscache_volume_get_hash_collision);
set_bit(FSCACHE_VOLUME_COLLIDED_WITH, &cursor->flags);
set_bit(FSCACHE_VOLUME_ACQUIRE_PENDING, &candidate->flags);
collidee_debug_id = cursor->debug_id;
break;
}
}
hlist_bl_add_head(&candidate->hash_link, h);
hlist_bl_unlock(h);
if (test_bit(FSCACHE_VOLUME_ACQUIRE_PENDING, &candidate->flags))
fscache_wait_on_volume_collision(candidate, collidee_debug_id);
return true;
collision:
fscache_see_volume(cursor, fscache_volume_collision);
hlist_bl_unlock(h);
return false;
}
/*
* Allocate and initialise a volume representation cookie.
*/
static struct fscache_volume *fscache_alloc_volume(const char *volume_key,
const char *cache_name,
const void *coherency_data,
size_t coherency_len)
{
struct fscache_volume *volume;
struct fscache_cache *cache;
size_t klen, hlen;
char *key;
if (!coherency_data)
coherency_len = 0;
cache = fscache_lookup_cache(cache_name, false);
if (IS_ERR(cache))
return NULL;
volume = kzalloc(struct_size(volume, coherency, coherency_len),
GFP_KERNEL);
if (!volume)
goto err_cache;
volume->cache = cache;
volume->coherency_len = coherency_len;
if (coherency_data)
memcpy(volume->coherency, coherency_data, coherency_len);
INIT_LIST_HEAD(&volume->proc_link);
INIT_WORK(&volume->work, fscache_create_volume_work);
refcount_set(&volume->ref, 1);
spin_lock_init(&volume->lock);
/* Stick the length on the front of the key and pad it out to make
* hashing easier.
*/
klen = strlen(volume_key);
hlen = round_up(1 + klen + 1, sizeof(__le32));
key = kzalloc(hlen, GFP_KERNEL);
if (!key)
goto err_vol;
key[0] = klen;
memcpy(key + 1, volume_key, klen);
volume->key = key;
volume->key_hash = fscache_hash(0, key, hlen);
volume->debug_id = atomic_inc_return(&fscache_volume_debug_id);
down_write(&fscache_addremove_sem);
atomic_inc(&cache->n_volumes);
list_add_tail(&volume->proc_link, &fscache_volumes);
fscache_see_volume(volume, fscache_volume_new_acquire);
fscache_stat(&fscache_n_volumes);
up_write(&fscache_addremove_sem);
_leave(" = v=%x", volume->debug_id);
return volume;
err_vol:
kfree(volume);
err_cache:
fscache_put_cache(cache, fscache_cache_put_alloc_volume);
fscache_stat(&fscache_n_volumes_nomem);
return NULL;
}
/*
* Create a volume's representation on disk. Have a volume ref and a cache
* access we have to release.
*/
static void fscache_create_volume_work(struct work_struct *work)
{
const struct fscache_cache_ops *ops;
struct fscache_volume *volume =
container_of(work, struct fscache_volume, work);
fscache_see_volume(volume, fscache_volume_see_create_work);
ops = volume->cache->ops;
if (ops->acquire_volume)
ops->acquire_volume(volume);
fscache_end_cache_access(volume->cache,
fscache_access_acquire_volume_end);
clear_bit_unlock(FSCACHE_VOLUME_CREATING, &volume->flags);
wake_up_bit(&volume->flags, FSCACHE_VOLUME_CREATING);
fscache_put_volume(volume, fscache_volume_put_create_work);
}
/*
* Dispatch a worker thread to create a volume's representation on disk.
*/
void fscache_create_volume(struct fscache_volume *volume, bool wait)
{
if (test_and_set_bit(FSCACHE_VOLUME_CREATING, &volume->flags))
goto maybe_wait;
if (volume->cache_priv)
goto no_wait; /* We raced */
if (!fscache_begin_cache_access(volume->cache,
fscache_access_acquire_volume))
goto no_wait;
fscache_get_volume(volume, fscache_volume_get_create_work);
if (!schedule_work(&volume->work))
fscache_put_volume(volume, fscache_volume_put_create_work);
maybe_wait:
if (wait) {
fscache_see_volume(volume, fscache_volume_wait_create_work);
wait_on_bit(&volume->flags, FSCACHE_VOLUME_CREATING,
TASK_UNINTERRUPTIBLE);
}
return;
no_wait:
clear_bit_unlock(FSCACHE_VOLUME_CREATING, &volume->flags);
wake_up_bit(&volume->flags, FSCACHE_VOLUME_CREATING);
}
/*
* Acquire a volume representation cookie and link it to a (proposed) cache.
*/
struct fscache_volume *__fscache_acquire_volume(const char *volume_key,
const char *cache_name,
const void *coherency_data,
size_t coherency_len)
{
struct fscache_volume *volume;
volume = fscache_alloc_volume(volume_key, cache_name,
coherency_data, coherency_len);
if (!volume)
return ERR_PTR(-ENOMEM);
if (!fscache_hash_volume(volume)) {
fscache_put_volume(volume, fscache_volume_put_hash_collision);
return ERR_PTR(-EBUSY);
}
fscache_create_volume(volume, false);
return volume;
}
EXPORT_SYMBOL(__fscache_acquire_volume);
static void fscache_wake_pending_volume(struct fscache_volume *volume,
struct hlist_bl_head *h)
{
struct fscache_volume *cursor;
struct hlist_bl_node *p;
hlist_bl_for_each_entry(cursor, p, h, hash_link) {
if (fscache_volume_same(cursor, volume)) {
fscache_see_volume(cursor, fscache_volume_see_hash_wake);
clear_bit(FSCACHE_VOLUME_ACQUIRE_PENDING, &cursor->flags);
wake_up_bit(&cursor->flags, FSCACHE_VOLUME_ACQUIRE_PENDING);
return;
}
}
}
/*
* Remove a volume cookie from the hash table.
*/
static void fscache_unhash_volume(struct fscache_volume *volume)
{
struct hlist_bl_head *h;
unsigned int bucket;
bucket = volume->key_hash & (ARRAY_SIZE(fscache_volume_hash) - 1);
h = &fscache_volume_hash[bucket];
hlist_bl_lock(h);
hlist_bl_del(&volume->hash_link);
if (test_bit(FSCACHE_VOLUME_COLLIDED_WITH, &volume->flags))
fscache_wake_pending_volume(volume, h);
hlist_bl_unlock(h);
}
/*
* Drop a cache's volume attachments.
*/
static void fscache_free_volume(struct fscache_volume *volume)
{
struct fscache_cache *cache = volume->cache;
if (volume->cache_priv) {
__fscache_begin_volume_access(volume, NULL,
fscache_access_relinquish_volume);
if (volume->cache_priv)
cache->ops->free_volume(volume);
fscache_end_volume_access(volume, NULL,
fscache_access_relinquish_volume_end);
}
down_write(&fscache_addremove_sem);
list_del_init(&volume->proc_link);
atomic_dec(&volume->cache->n_volumes);
up_write(&fscache_addremove_sem);
if (!hlist_bl_unhashed(&volume->hash_link))
fscache_unhash_volume(volume);
trace_fscache_volume(volume->debug_id, 0, fscache_volume_free);
kfree(volume->key);
kfree(volume);
fscache_stat_d(&fscache_n_volumes);
fscache_put_cache(cache, fscache_cache_put_volume);
}
/*
* Drop a reference to a volume cookie.
*/
void fscache_put_volume(struct fscache_volume *volume,
enum fscache_volume_trace where)
{
if (volume) {
unsigned int debug_id = volume->debug_id;
bool zero;
int ref;
zero = __refcount_dec_and_test(&volume->ref, &ref);
trace_fscache_volume(debug_id, ref - 1, where);
if (zero)
fscache_free_volume(volume);
}
}
/*
* Relinquish a volume representation cookie.
*/
void __fscache_relinquish_volume(struct fscache_volume *volume,
const void *coherency_data,
bool invalidate)
{
if (WARN_ON(test_and_set_bit(FSCACHE_VOLUME_RELINQUISHED, &volume->flags)))
return;
if (invalidate) {
set_bit(FSCACHE_VOLUME_INVALIDATE, &volume->flags);
} else if (coherency_data) {
memcpy(volume->coherency, coherency_data, volume->coherency_len);
}
fscache_put_volume(volume, fscache_volume_put_relinquish);
}
EXPORT_SYMBOL(__fscache_relinquish_volume);
/**
* fscache_withdraw_volume - Withdraw a volume from being cached
* @volume: Volume cookie
*
* Withdraw a cache volume from service, waiting for all accesses to complete
* before returning.
*/
void fscache_withdraw_volume(struct fscache_volume *volume)
{
int n_accesses;
_debug("withdraw V=%x", volume->debug_id);
/* Allow wakeups on dec-to-0 */
n_accesses = atomic_dec_return(&volume->n_accesses);
trace_fscache_access_volume(volume->debug_id, 0,
refcount_read(&volume->ref),
n_accesses, fscache_access_cache_unpin);
wait_var_event(&volume->n_accesses,
atomic_read(&volume->n_accesses) == 0);
}
EXPORT_SYMBOL(fscache_withdraw_volume);
#ifdef CONFIG_PROC_FS
/*
* Generate a list of volumes in /proc/fs/fscache/volumes
*/
static int fscache_volumes_seq_show(struct seq_file *m, void *v)
{
struct fscache_volume *volume;
if (v == &fscache_volumes) {
seq_puts(m,
"VOLUME REF nCOOK ACC FL CACHE KEY\n"
"======== ===== ===== === == =============== ================\n");
return 0;
}
volume = list_entry(v, struct fscache_volume, proc_link);
seq_printf(m,
"%08x %5d %5d %3d %02lx %-15.15s %s\n",
volume->debug_id,
refcount_read(&volume->ref),
atomic_read(&volume->n_cookies),
atomic_read(&volume->n_accesses),
volume->flags,
volume->cache->name ?: "-",
volume->key + 1);
return 0;
}
static void *fscache_volumes_seq_start(struct seq_file *m, loff_t *_pos)
__acquires(&fscache_addremove_sem)
{
down_read(&fscache_addremove_sem);
return seq_list_start_head(&fscache_volumes, *_pos);
}
static void *fscache_volumes_seq_next(struct seq_file *m, void *v, loff_t *_pos)
{
return seq_list_next(v, &fscache_volumes, _pos);
}
static void fscache_volumes_seq_stop(struct seq_file *m, void *v)
__releases(&fscache_addremove_sem)
{
up_read(&fscache_addremove_sem);
}
const struct seq_operations fscache_volumes_seq_ops = {
.start = fscache_volumes_seq_start,
.next = fscache_volumes_seq_next,
.stop = fscache_volumes_seq_stop,
.show = fscache_volumes_seq_show,
};
#endif /* CONFIG_PROC_FS */

3
fs/namei.c
View File

@ -3958,7 +3958,8 @@ int vfs_rmdir(struct user_namespace *mnt_userns, struct inode *dir,
inode_lock(dentry->d_inode);
error = -EBUSY;
if (is_local_mountpoint(dentry))
if (is_local_mountpoint(dentry) ||
(dentry->d_inode->i_flags & S_KERNEL_FILE))
goto out;
error = security_inode_rmdir(dir, dentry);

10
fs/netfs/read_helper.c
View File

@ -170,7 +170,7 @@ static void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error
*/
static void netfs_read_from_cache(struct netfs_read_request *rreq,
struct netfs_read_subrequest *subreq,
bool seek_data)
enum netfs_read_from_hole read_hole)
{
struct netfs_cache_resources *cres = &rreq->cache_resources;
struct iov_iter iter;
@ -180,7 +180,7 @@ static void netfs_read_from_cache(struct netfs_read_request *rreq,
subreq->start + subreq->transferred,
subreq->len - subreq->transferred);
cres->ops->read(cres, subreq->start, &iter, seek_data,
cres->ops->read(cres, subreq->start, &iter, read_hole,
netfs_cache_read_terminated, subreq);
}
@ -323,7 +323,7 @@ static void netfs_rreq_do_write_to_cache(struct netfs_read_request *rreq)
}
ret = cres->ops->prepare_write(cres, &subreq->start, &subreq->len,
rreq->i_size);
rreq->i_size, true);
if (ret < 0) {
trace_netfs_failure(rreq, subreq, ret, netfs_fail_prepare_write);
trace_netfs_sreq(subreq, netfs_sreq_trace_write_skip);
@ -461,7 +461,7 @@ static void netfs_rreq_short_read(struct netfs_read_request *rreq,
netfs_get_read_subrequest(subreq);
atomic_inc(&rreq->nr_rd_ops);
if (subreq->source == NETFS_READ_FROM_CACHE)
netfs_read_from_cache(rreq, subreq, true);
netfs_read_from_cache(rreq, subreq, NETFS_READ_HOLE_CLEAR);
else
netfs_read_from_server(rreq, subreq);
}
@ -789,7 +789,7 @@ static bool netfs_rreq_submit_slice(struct netfs_read_request *rreq,
netfs_read_from_server(rreq, subreq);
break;
case NETFS_READ_FROM_CACHE:
netfs_read_from_cache(rreq, subreq, false);
netfs_read_from_cache(rreq, subreq, NETFS_READ_HOLE_IGNORE);
break;
default:
BUG();

2
fs/nfs/Makefile
View File

@ -12,7 +12,7 @@ nfs-y := client.o dir.o file.o getroot.o inode.o super.o \
export.o sysfs.o fs_context.o
nfs-$(CONFIG_ROOT_NFS) += nfsroot.o
nfs-$(CONFIG_SYSCTL) += sysctl.o
nfs-$(CONFIG_NFS_FSCACHE) += fscache.o fscache-index.o
nfs-$(CONFIG_NFS_FSCACHE) += fscache.o
obj-$(CONFIG_NFS_V2) += nfsv2.o
nfsv2-y := nfs2super.o proc.o nfs2xdr.o

4
fs/nfs/client.c
View File

@ -183,8 +183,6 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
clp->cl_net = get_net(cl_init->net);
clp->cl_principal = "*";
nfs_fscache_get_client_cookie(clp);
return clp;
error_cleanup:
@ -238,8 +236,6 @@ static void pnfs_init_server(struct nfs_server *server)
*/
void nfs_free_client(struct nfs_client *clp)
{
nfs_fscache_release_client_cookie(clp);
/* -EIO all pending I/O */
if (!IS_ERR(clp->cl_rpcclient))
rpc_shutdown_client(clp->cl_rpcclient);

2
fs/nfs/direct.c
View File

@ -59,6 +59,7 @@
#include "internal.h"
#include "iostat.h"
#include "pnfs.h"
#include "fscache.h"
#define NFSDBG_FACILITY NFSDBG_VFS
@ -959,6 +960,7 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter)
} else {
result = requested;
}
nfs_fscache_invalidate(inode, FSCACHE_INVAL_DIO_WRITE);
out_release:
nfs_direct_req_release(dreq);
out:

13
fs/nfs/file.c
View File

@ -84,6 +84,7 @@ nfs_file_release(struct inode *inode, struct file *filp)
nfs_inc_stats(inode, NFSIOS_VFSRELEASE);
nfs_file_clear_open_context(filp);
nfs_fscache_release_file(inode, filp);
return 0;
}
EXPORT_SYMBOL_GPL(nfs_file_release);
@ -415,8 +416,7 @@ static void nfs_invalidate_page(struct page *page, unsigned int offset,
return;
/* Cancel any unstarted writes on this page */
nfs_wb_page_cancel(page_file_mapping(page)->host, page);
nfs_fscache_invalidate_page(page, page->mapping->host);
wait_on_page_fscache(page);
}
/*
@ -475,12 +475,11 @@ static void nfs_check_dirty_writeback(struct page *page,
static int nfs_launder_page(struct page *page)
{
struct inode *inode = page_file_mapping(page)->host;
struct nfs_inode *nfsi = NFS_I(inode);
dfprintk(PAGECACHE, "NFS: launder_page(%ld, %llu)\n",
inode->i_ino, (long long)page_offset(page));
nfs_fscache_wait_on_page_write(nfsi, page);
wait_on_page_fscache(page);
return nfs_wb_page(inode, page);
}
@ -555,7 +554,11 @@ static vm_fault_t nfs_vm_page_mkwrite(struct vm_fault *vmf)
sb_start_pagefault(inode->i_sb);
/* make sure the cache has finished storing the page */
nfs_fscache_wait_on_page_write(NFS_I(inode), page);
if (PageFsCache(page) &&
wait_on_page_fscache_killable(vmf->page) < 0) {
ret = VM_FAULT_RETRY;
goto out;
}
wait_on_bit_action(&NFS_I(inode)->flags, NFS_INO_INVALIDATING,
nfs_wait_bit_killable, TASK_KILLABLE);

140
fs/nfs/fscache-index.c
View File

@ -1,140 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* NFS FS-Cache index structure definition
*
* Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/mm.h>
#include <linux/nfs_fs.h>
#include <linux/nfs_fs_sb.h>
#include <linux/in6.h>
#include <linux/iversion.h>
#include "internal.h"
#include "fscache.h"
#define NFSDBG_FACILITY NFSDBG_FSCACHE
/*
* Define the NFS filesystem for FS-Cache. Upon registration FS-Cache sticks
* the cookie for the top-level index object for NFS into here. The top-level
* index can than have other cache objects inserted into it.
*/
struct fscache_netfs nfs_fscache_netfs = {
.name = "nfs",
.version = 0,
};
/*
* Register NFS for caching
*/
int nfs_fscache_register(void)
{
return fscache_register_netfs(&nfs_fscache_netfs);
}
/*
* Unregister NFS for caching
*/
void nfs_fscache_unregister(void)
{
fscache_unregister_netfs(&nfs_fscache_netfs);
}
/*
* Define the server object for FS-Cache. This is used to describe a server
* object to fscache_acquire_cookie(). It is keyed by the NFS protocol and
* server address parameters.
*/
const struct fscache_cookie_def nfs_fscache_server_index_def = {
.name = "NFS.server",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
/*
* Define the superblock object for FS-Cache. This is used to describe a
* superblock object to fscache_acquire_cookie(). It is keyed by all the NFS
* parameters that might cause a separate superblock.
*/
const struct fscache_cookie_def nfs_fscache_super_index_def = {
.name = "NFS.super",
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
/*
* Consult the netfs about the state of an object
* - This function can be absent if the index carries no state data
* - The netfs data from the cookie being used as the target is
* presented, as is the auxiliary data
*/
static
enum fscache_checkaux nfs_fscache_inode_check_aux(void *cookie_netfs_data,
const void *data,
uint16_t datalen,
loff_t object_size)
{
struct nfs_fscache_inode_auxdata auxdata;
struct nfs_inode *nfsi = cookie_netfs_data;
if (datalen != sizeof(auxdata))
return FSCACHE_CHECKAUX_OBSOLETE;
memset(&auxdata, 0, sizeof(auxdata));
auxdata.mtime_sec = nfsi->vfs_inode.i_mtime.tv_sec;
auxdata.mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
auxdata.ctime_sec = nfsi->vfs_inode.i_ctime.tv_sec;
auxdata.ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
auxdata.change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
if (memcmp(data, &auxdata, datalen) != 0)
return FSCACHE_CHECKAUX_OBSOLETE;
return FSCACHE_CHECKAUX_OKAY;
}
/*
* Get an extra reference on a read context.
* - This function can be absent if the completion function doesn't require a
* context.
* - The read context is passed back to NFS in the event that a data read on the
* cache fails with EIO - in which case the server must be contacted to
* retrieve the data, which requires the read context for security.
*/
static void nfs_fh_get_context(void *cookie_netfs_data, void *context)
{
get_nfs_open_context(context);
}
/*
* Release an extra reference on a read context.
* - This function can be absent if the completion function doesn't require a
* context.
*/
static void nfs_fh_put_context(void *cookie_netfs_data, void *context)
{
if (context)
put_nfs_open_context(context);
}
/*
* Define the inode object for FS-Cache. This is used to describe an inode
* object to fscache_acquire_cookie(). It is keyed by the NFS file handle for
* an inode.
*
* Coherency is managed by comparing the copies of i_size, i_mtime and i_ctime
* held in the cache auxiliary data for the data storage object with those in
* the inode struct in memory.
*/
const struct fscache_cookie_def nfs_fscache_inode_object_def = {
.name = "NFS.fh",
.type = FSCACHE_COOKIE_TYPE_DATAFILE,
.check_aux = nfs_fscache_inode_check_aux,
.get_context = nfs_fh_get_context,
.put_context = nfs_fh_put_context,
};

504
fs/nfs/fscache.c
View File

@ -22,24 +22,18 @@
#define NFSDBG_FACILITY NFSDBG_FSCACHE
static struct rb_root nfs_fscache_keys = RB_ROOT;
static DEFINE_SPINLOCK(nfs_fscache_keys_lock);
#define NFS_MAX_KEY_LEN 1000
/*
* Layout of the key for an NFS server cache object.
*/
struct nfs_server_key {
struct {
uint16_t nfsversion; /* NFS protocol version */
uint32_t minorversion; /* NFSv4 minor version */
uint16_t family; /* address family */
__be16 port; /* IP port */
} hdr;
union {
struct in_addr ipv4_addr; /* IPv4 address */
struct in6_addr ipv6_addr; /* IPv6 address */
};
} __packed;
static bool nfs_append_int(char *key, int *_len, unsigned long long x)
{
if (*_len > NFS_MAX_KEY_LEN)
return false;
if (x == 0)
key[(*_len)++] = ',';
else
*_len += sprintf(key + *_len, ",%llx", x);
return true;
}
/*
* Get the per-client index cookie for an NFS client if the appropriate mount
@ -47,160 +41,108 @@ struct nfs_server_key {
* - We always try and get an index cookie for the client, but get filehandle
* cookies on a per-superblock basis, depending on the mount flags
*/
void nfs_fscache_get_client_cookie(struct nfs_client *clp)
static bool nfs_fscache_get_client_key(struct nfs_client *clp,
char *key, int *_len)
{
const struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *) &clp->cl_addr;
const struct sockaddr_in *sin = (struct sockaddr_in *) &clp->cl_addr;
struct nfs_server_key key;
uint16_t len = sizeof(key.hdr);
memset(&key, 0, sizeof(key));
key.hdr.nfsversion = clp->rpc_ops->version;
key.hdr.minorversion = clp->cl_minorversion;
key.hdr.family = clp->cl_addr.ss_family;
*_len += snprintf(key + *_len, NFS_MAX_KEY_LEN - *_len,
",%u.%u,%x",
clp->rpc_ops->version,
clp->cl_minorversion,
clp->cl_addr.ss_family);
switch (clp->cl_addr.ss_family) {
case AF_INET:
key.hdr.port = sin->sin_port;
key.ipv4_addr = sin->sin_addr;
len += sizeof(key.ipv4_addr);
break;
if (!nfs_append_int(key, _len, sin->sin_port) ||
!nfs_append_int(key, _len, sin->sin_addr.s_addr))
return false;
return true;
case AF_INET6:
key.hdr.port = sin6->sin6_port;
key.ipv6_addr = sin6->sin6_addr;
len += sizeof(key.ipv6_addr);
break;
if (!nfs_append_int(key, _len, sin6->sin6_port) ||
!nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[0]) ||
!nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[1]) ||
!nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[2]) ||
!nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[3]))
return false;
return true;
default:
printk(KERN_WARNING "NFS: Unknown network family '%d'\n",
clp->cl_addr.ss_family);
clp->fscache = NULL;
return;
return false;
}
/* create a cache index for looking up filehandles */
clp->fscache = fscache_acquire_cookie(nfs_fscache_netfs.primary_index,
&nfs_fscache_server_index_def,
&key, len,
NULL, 0,
clp, 0, true);
dfprintk(FSCACHE, "NFS: get client cookie (0x%p/0x%p)\n",
clp, clp->fscache);
}
/*
* Dispose of a per-client cookie
*/
void nfs_fscache_release_client_cookie(struct nfs_client *clp)
{
dfprintk(FSCACHE, "NFS: releasing client cookie (0x%p/0x%p)\n",
clp, clp->fscache);
fscache_relinquish_cookie(clp->fscache, NULL, false);
clp->fscache = NULL;
}
/*
* Get the cache cookie for an NFS superblock. We have to handle
* uniquification here because the cache doesn't do it for us.
* Get the cache cookie for an NFS superblock.
*
* The default uniquifier is just an empty string, but it may be overridden
* either by the 'fsc=xxx' option to mount, or by inheriting it from the parent
* superblock across an automount point of some nature.
*/
void nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int ulen)
int nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int ulen)
{
struct nfs_fscache_key *key, *xkey;
struct fscache_volume *vcookie;
struct nfs_server *nfss = NFS_SB(sb);
struct rb_node **p, *parent;
int diff;
unsigned int len = 3;
char *key;
nfss->fscache_key = NULL;
nfss->fscache = NULL;
if (!uniq) {
uniq = "";
ulen = 1;
if (uniq) {
nfss->fscache_uniq = kmemdup_nul(uniq, ulen, GFP_KERNEL);
if (!nfss->fscache_uniq)
return -ENOMEM;
}
key = kzalloc(sizeof(*key) + ulen, GFP_KERNEL);
key = kmalloc(NFS_MAX_KEY_LEN + 24, GFP_KERNEL);
if (!key)
return;
return -ENOMEM;
key->nfs_client = nfss->nfs_client;
key->key.super.s_flags = sb->s_flags & NFS_SB_MASK;
key->key.nfs_server.flags = nfss->flags;
key->key.nfs_server.rsize = nfss->rsize;
key->key.nfs_server.wsize = nfss->wsize;
key->key.nfs_server.acregmin = nfss->acregmin;
key->key.nfs_server.acregmax = nfss->acregmax;
key->key.nfs_server.acdirmin = nfss->acdirmin;
key->key.nfs_server.acdirmax = nfss->acdirmax;
key->key.nfs_server.fsid = nfss->fsid;
key->key.rpc_auth.au_flavor = nfss->client->cl_auth->au_flavor;
memcpy(key, "nfs", 3);
if (!nfs_fscache_get_client_key(nfss->nfs_client, key, &len) ||
!nfs_append_int(key, &len, nfss->fsid.major) ||
!nfs_append_int(key, &len, nfss->fsid.minor) ||
!nfs_append_int(key, &len, sb->s_flags & NFS_SB_MASK) ||
!nfs_append_int(key, &len, nfss->flags) ||
!nfs_append_int(key, &len, nfss->rsize) ||
!nfs_append_int(key, &len, nfss->wsize) ||
!nfs_append_int(key, &len, nfss->acregmin) ||
!nfs_append_int(key, &len, nfss->acregmax) ||
!nfs_append_int(key, &len, nfss->acdirmin) ||
!nfs_append_int(key, &len, nfss->acdirmax) ||
!nfs_append_int(key, &len, nfss->client->cl_auth->au_flavor))
goto out;
key->key.uniq_len = ulen;
memcpy(key->key.uniquifier, uniq, ulen);
spin_lock(&nfs_fscache_keys_lock);
p = &nfs_fscache_keys.rb_node;
parent = NULL;
while (*p) {
parent = *p;
xkey = rb_entry(parent, struct nfs_fscache_key, node);
if (key->nfs_client < xkey->nfs_client)
goto go_left;
if (key->nfs_client > xkey->nfs_client)
goto go_right;
diff = memcmp(&key->key, &xkey->key, sizeof(key->key));
if (diff < 0)
goto go_left;
if (diff > 0)
goto go_right;
if (key->key.uniq_len == 0)
goto non_unique;
diff = memcmp(key->key.uniquifier,
xkey->key.uniquifier,
key->key.uniq_len);
if (diff < 0)
goto go_left;
if (diff > 0)
goto go_right;
goto non_unique;
go_left:
p = &(*p)->rb_left;
continue;
go_right:
p = &(*p)->rb_right;
if (ulen > 0) {
if (ulen > NFS_MAX_KEY_LEN - len)
goto out;
key[len++] = ',';
memcpy(key + len, uniq, ulen);
len += ulen;
}
rb_link_node(&key->node, parent, p);
rb_insert_color(&key->node, &nfs_fscache_keys);
spin_unlock(&nfs_fscache_keys_lock);
nfss->fscache_key = key;
key[len] = 0;
/* create a cache index for looking up filehandles */
nfss->fscache = fscache_acquire_cookie(nfss->nfs_client->fscache,
&nfs_fscache_super_index_def,
&key->key,
sizeof(key->key) + ulen,
NULL, 0,
nfss, 0, true);
vcookie = fscache_acquire_volume(key,
NULL, /* preferred_cache */
NULL, 0 /* coherency_data */);
dfprintk(FSCACHE, "NFS: get superblock cookie (0x%p/0x%p)\n",
nfss, nfss->fscache);
return;
nfss, vcookie);
if (IS_ERR(vcookie)) {
if (vcookie != ERR_PTR(-EBUSY)) {
kfree(key);
return PTR_ERR(vcookie);
}
pr_err("NFS: Cache volume key already in use (%s)\n", key);
vcookie = NULL;
}
nfss->fscache = vcookie;
non_unique:
spin_unlock(&nfs_fscache_keys_lock);
out:
kfree(key);
nfss->fscache_key = NULL;
nfss->fscache = NULL;
printk(KERN_WARNING "NFS:"
" Cache request denied due to non-unique superblock keys\n");
return 0;
}
/*
@ -213,29 +155,9 @@ void nfs_fscache_release_super_cookie(struct super_block *sb)
dfprintk(FSCACHE, "NFS: releasing superblock cookie (0x%p/0x%p)\n",
nfss, nfss->fscache);
fscache_relinquish_cookie(nfss->fscache, NULL, false);
fscache_relinquish_volume(nfss->fscache, NULL, false);
nfss->fscache = NULL;
if (nfss->fscache_key) {
spin_lock(&nfs_fscache_keys_lock);
rb_erase(&nfss->fscache_key->node, &nfs_fscache_keys);
spin_unlock(&nfs_fscache_keys_lock);
kfree(nfss->fscache_key);
nfss->fscache_key = NULL;
}
}
static void nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
struct nfs_inode *nfsi)
{
memset(auxdata, 0, sizeof(*auxdata));
auxdata->mtime_sec = nfsi->vfs_inode.i_mtime.tv_sec;
auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
auxdata->ctime_sec = nfsi->vfs_inode.i_ctime.tv_sec;
auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
kfree(nfss->fscache_uniq);
}
/*
@ -254,10 +176,12 @@ void nfs_fscache_init_inode(struct inode *inode)
nfs_fscache_update_auxdata(&auxdata, nfsi);
nfsi->fscache = fscache_acquire_cookie(NFS_SB(inode->i_sb)->fscache,
&nfs_fscache_inode_object_def,
nfsi->fh.data, nfsi->fh.size,
&auxdata, sizeof(auxdata),
nfsi, nfsi->vfs_inode.i_size, false);
0,
nfsi->fh.data, /* index_key */
nfsi->fh.size,
&auxdata, /* aux_data */
sizeof(auxdata),
i_size_read(&nfsi->vfs_inode));
}
/*
@ -265,24 +189,15 @@ void nfs_fscache_init_inode(struct inode *inode)
*/
void nfs_fscache_clear_inode(struct inode *inode)
{
struct nfs_fscache_inode_auxdata auxdata;
struct nfs_inode *nfsi = NFS_I(inode);
struct fscache_cookie *cookie = nfs_i_fscache(inode);
dfprintk(FSCACHE, "NFS: clear cookie (0x%p/0x%p)\n", nfsi, cookie);
nfs_fscache_update_auxdata(&auxdata, nfsi);
fscache_relinquish_cookie(cookie, &auxdata, false);
fscache_relinquish_cookie(cookie, false);
nfsi->fscache = NULL;
}
static bool nfs_fscache_can_enable(void *data)
{
struct inode *inode = data;
return !inode_is_open_for_write(inode);
}
/*
* Enable or disable caching for a file that is being opened as appropriate.
* The cookie is allocated when the inode is initialised, but is not enabled at
@ -307,100 +222,104 @@ void nfs_fscache_open_file(struct inode *inode, struct file *filp)
struct nfs_fscache_inode_auxdata auxdata;
struct nfs_inode *nfsi = NFS_I(inode);
struct fscache_cookie *cookie = nfs_i_fscache(inode);
bool open_for_write = inode_is_open_for_write(inode);
if (!fscache_cookie_valid(cookie))
return;
nfs_fscache_update_auxdata(&auxdata, nfsi);
if (inode_is_open_for_write(inode)) {
fscache_use_cookie(cookie, open_for_write);
if (open_for_write) {
dfprintk(FSCACHE, "NFS: nfsi 0x%p disabling cache\n", nfsi);
clear_bit(NFS_INO_FSCACHE, &nfsi->flags);
fscache_disable_cookie(cookie, &auxdata, true);
fscache_uncache_all_inode_pages(cookie, inode);
} else {
dfprintk(FSCACHE, "NFS: nfsi 0x%p enabling cache\n", nfsi);
fscache_enable_cookie(cookie, &auxdata, nfsi->vfs_inode.i_size,
nfs_fscache_can_enable, inode);
if (fscache_cookie_enabled(cookie))
set_bit(NFS_INO_FSCACHE, &NFS_I(inode)->flags);
nfs_fscache_update_auxdata(&auxdata, nfsi);
fscache_invalidate(cookie, &auxdata, i_size_read(inode),
FSCACHE_INVAL_DIO_WRITE);
}
}
EXPORT_SYMBOL_GPL(nfs_fscache_open_file);
/*
* Release the caching state associated with a page, if the page isn't busy
* interacting with the cache.
* - Returns true (can release page) or false (page busy).
*/
int nfs_fscache_release_page(struct page *page, gfp_t gfp)
{
if (PageFsCache(page)) {
struct fscache_cookie *cookie = nfs_i_fscache(page->mapping->host);
BUG_ON(!cookie);
dfprintk(FSCACHE, "NFS: fscache releasepage (0x%p/0x%p/0x%p)\n",
cookie, page, NFS_I(page->mapping->host));
if (!fscache_maybe_release_page(cookie, page, gfp))
return 0;
nfs_inc_fscache_stats(page->mapping->host,
NFSIOS_FSCACHE_PAGES_UNCACHED);
}
return 1;
}
/*
* Release the caching state associated with a page if undergoing complete page
* invalidation.
*/
void __nfs_fscache_invalidate_page(struct page *page, struct inode *inode)
void nfs_fscache_release_file(struct inode *inode, struct file *filp)
{
struct nfs_fscache_inode_auxdata auxdata;
struct nfs_inode *nfsi = NFS_I(inode);
struct fscache_cookie *cookie = nfs_i_fscache(inode);
BUG_ON(!cookie);
if (fscache_cookie_valid(cookie)) {
nfs_fscache_update_auxdata(&auxdata, nfsi);
fscache_unuse_cookie(cookie, &auxdata, NULL);
}
}
dfprintk(FSCACHE, "NFS: fscache invalidatepage (0x%p/0x%p/0x%p)\n",
cookie, page, NFS_I(inode));
static inline void fscache_end_operation(struct netfs_cache_resources *cres)
{
const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
fscache_wait_on_page_write(cookie, page);
BUG_ON(!PageLocked(page));
fscache_uncache_page(cookie, page);
nfs_inc_fscache_stats(page->mapping->host,
NFSIOS_FSCACHE_PAGES_UNCACHED);
if (ops)
ops->end_operation(cres);
}
/*
* Handle completion of a page being read from the cache.
* - Called in process (keventd) context.
* Fallback page reading interface.
*/
static void nfs_readpage_from_fscache_complete(struct page *page,
void *context,
int error)
static int fscache_fallback_read_page(struct inode *inode, struct page *page)
{
dfprintk(FSCACHE,
"NFS: readpage_from_fscache_complete (0x%p/0x%p/%d)\n",
page, context, error);
struct netfs_cache_resources cres;
struct fscache_cookie *cookie = nfs_i_fscache(inode);
struct iov_iter iter;
struct bio_vec bvec[1];
int ret;
/*
* If the read completes with an error, mark the page with PG_checked,
* unlock the page, and let the VM reissue the readpage.
*/
if (!error)
SetPageUptodate(page);
else
SetPageChecked(page);
unlock_page(page);
memset(&cres, 0, sizeof(cres));
bvec[0].bv_page = page;
bvec[0].bv_offset = 0;
bvec[0].bv_len = PAGE_SIZE;
iov_iter_bvec(&iter, READ, bvec, ARRAY_SIZE(bvec), PAGE_SIZE);
ret = fscache_begin_read_operation(&cres, cookie);
if (ret < 0)
return ret;
ret = fscache_read(&cres, page_offset(page), &iter, NETFS_READ_HOLE_FAIL,
NULL, NULL);
fscache_end_operation(&cres);
return ret;
}
/*
* Fallback page writing interface.
*/
static int fscache_fallback_write_page(struct inode *inode, struct page *page,
bool no_space_allocated_yet)
{
struct netfs_cache_resources cres;
struct fscache_cookie *cookie = nfs_i_fscache(inode);
struct iov_iter iter;
struct bio_vec bvec[1];
loff_t start = page_offset(page);
size_t len = PAGE_SIZE;
int ret;
memset(&cres, 0, sizeof(cres));
bvec[0].bv_page = page;
bvec[0].bv_offset = 0;
bvec[0].bv_len = PAGE_SIZE;
iov_iter_bvec(&iter, WRITE, bvec, ARRAY_SIZE(bvec), PAGE_SIZE);
ret = fscache_begin_write_operation(&cres, cookie);
if (ret < 0)
return ret;
ret = cres.ops->prepare_write(&cres, &start, &len, i_size_read(inode),
no_space_allocated_yet);
if (ret == 0)
ret = fscache_write(&cres, page_offset(page), &iter, NULL, NULL);
fscache_end_operation(&cres);
return ret;
}
/*
* Retrieve a page from fscache
*/
int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,
struct inode *inode, struct page *page)
int __nfs_readpage_from_fscache(struct inode *inode, struct page *page)
{
int ret;
@ -409,112 +328,49 @@ int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,
nfs_i_fscache(inode), page, page->index, page->flags, inode);
if (PageChecked(page)) {
dfprintk(FSCACHE, "NFS: readpage_from_fscache: PageChecked\n");
ClearPageChecked(page);
return 1;
}
ret = fscache_read_or_alloc_page(nfs_i_fscache(inode),
page,
nfs_readpage_from_fscache_complete,
ctx,
GFP_KERNEL);
switch (ret) {
case 0: /* read BIO submitted (page in fscache) */
ret = fscache_fallback_read_page(inode, page);
if (ret < 0) {
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
dfprintk(FSCACHE,
"NFS: readpage_from_fscache: BIO submitted\n");
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK);
"NFS: readpage_from_fscache failed %d\n", ret);
SetPageChecked(page);
return ret;
case -ENOBUFS: /* inode not in cache */
case -ENODATA: /* page not in cache */
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
dfprintk(FSCACHE,
"NFS: readpage_from_fscache %d\n", ret);
return 1;
default:
dfprintk(FSCACHE, "NFS: readpage_from_fscache %d\n", ret);
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
}
return ret;
/* Read completed synchronously */
dfprintk(FSCACHE, "NFS: readpage_from_fscache: read successful\n");
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK);
SetPageUptodate(page);
return 0;
}
/*
* Retrieve a set of pages from fscache
* Store a newly fetched page in fscache. We can be certain there's no page
* stored in the cache as yet otherwise we would've read it from there.
*/
int __nfs_readpages_from_fscache(struct nfs_open_context *ctx,
struct inode *inode,
struct address_space *mapping,
struct list_head *pages,
unsigned *nr_pages)
{
unsigned npages = *nr_pages;
int ret;
dfprintk(FSCACHE, "NFS: nfs_getpages_from_fscache (0x%p/%u/0x%p)\n",
nfs_i_fscache(inode), npages, inode);
ret = fscache_read_or_alloc_pages(nfs_i_fscache(inode),
mapping, pages, nr_pages,
nfs_readpage_from_fscache_complete,
ctx,
mapping_gfp_mask(mapping));
if (*nr_pages < npages)
nfs_add_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK,
npages);
if (*nr_pages > 0)
nfs_add_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL,
*nr_pages);
switch (ret) {
case 0: /* read submitted to the cache for all pages */
BUG_ON(!list_empty(pages));
BUG_ON(*nr_pages != 0);
dfprintk(FSCACHE,
"NFS: nfs_getpages_from_fscache: submitted\n");
return ret;
case -ENOBUFS: /* some pages aren't cached and can't be */
case -ENODATA: /* some pages aren't cached */
dfprintk(FSCACHE,
"NFS: nfs_getpages_from_fscache: no page: %d\n", ret);
return 1;
default:
dfprintk(FSCACHE,
"NFS: nfs_getpages_from_fscache: ret %d\n", ret);
}
return ret;
}
/*
* Store a newly fetched page in fscache
* - PG_fscache must be set on the page
*/
void __nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
void __nfs_readpage_to_fscache(struct inode *inode, struct page *page)
{
int ret;
dfprintk(FSCACHE,
"NFS: readpage_to_fscache(fsc:%p/p:%p(i:%lx f:%lx)/%d)\n",
nfs_i_fscache(inode), page, page->index, page->flags, sync);
"NFS: readpage_to_fscache(fsc:%p/p:%p(i:%lx f:%lx))\n",
nfs_i_fscache(inode), page, page->index, page->flags);
ret = fscache_fallback_write_page(inode, page, true);
ret = fscache_write_page(nfs_i_fscache(inode), page,
inode->i_size, GFP_KERNEL);
dfprintk(FSCACHE,
"NFS: readpage_to_fscache: p:%p(i:%lu f:%lx) ret %d\n",
page, page->index, page->flags, ret);
if (ret != 0) {
fscache_uncache_page(nfs_i_fscache(inode), page);
nfs_inc_fscache_stats(inode,
NFSIOS_FSCACHE_PAGES_WRITTEN_FAIL);
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_WRITTEN_FAIL);
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_UNCACHED);
} else {
nfs_inc_fscache_stats(inode,
NFSIOS_FSCACHE_PAGES_WRITTEN_OK);
nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_WRITTEN_OK);
}
}

182
fs/nfs/fscache.h
View File

@ -8,50 +8,15 @@
#ifndef _NFS_FSCACHE_H
#define _NFS_FSCACHE_H
#include <linux/swap.h>
#include <linux/nfs_fs.h>
#include <linux/nfs_mount.h>
#include <linux/nfs4_mount.h>
#include <linux/fscache.h>
#include <linux/iversion.h>
#ifdef CONFIG_NFS_FSCACHE
/*
* set of NFS FS-Cache objects that form a superblock key
*/
struct nfs_fscache_key {
struct rb_node node;
struct nfs_client *nfs_client; /* the server */
/* the elements of the unique key - as used by nfs_compare_super() and
* nfs_compare_mount_options() to distinguish superblocks */
struct {
struct {
unsigned long s_flags; /* various flags
* (& NFS_MS_MASK) */
} super;
struct {
struct nfs_fsid fsid;
int flags;
unsigned int rsize; /* read size */
unsigned int wsize; /* write size */
unsigned int acregmin; /* attr cache timeouts */
unsigned int acregmax;
unsigned int acdirmin;
unsigned int acdirmax;
} nfs_server;
struct {
rpc_authflavor_t au_flavor;
} rpc_auth;
/* uniquifier - can be used if nfs_server.flags includes
* NFS_MOUNT_UNSHARED */
u8 uniq_len;
char uniquifier[0];
} key;
};
/*
* Definition of the auxiliary data attached to NFS inode storage objects
* within the cache.
@ -69,85 +34,43 @@ struct nfs_fscache_inode_auxdata {
u64 change_attr;
};
/*
* fscache-index.c
*/
extern struct fscache_netfs nfs_fscache_netfs;
extern const struct fscache_cookie_def nfs_fscache_server_index_def;
extern const struct fscache_cookie_def nfs_fscache_super_index_def;
extern const struct fscache_cookie_def nfs_fscache_inode_object_def;
extern int nfs_fscache_register(void);
extern void nfs_fscache_unregister(void);
/*
* fscache.c
*/
extern void nfs_fscache_get_client_cookie(struct nfs_client *);
extern void nfs_fscache_release_client_cookie(struct nfs_client *);
extern void nfs_fscache_get_super_cookie(struct super_block *, const char *, int);
extern int nfs_fscache_get_super_cookie(struct super_block *, const char *, int);
extern void nfs_fscache_release_super_cookie(struct super_block *);
extern void nfs_fscache_init_inode(struct inode *);
extern void nfs_fscache_clear_inode(struct inode *);
extern void nfs_fscache_open_file(struct inode *, struct file *);
extern void nfs_fscache_release_file(struct inode *, struct file *);
extern void __nfs_fscache_invalidate_page(struct page *, struct inode *);
extern int nfs_fscache_release_page(struct page *, gfp_t);
extern int __nfs_readpage_from_fscache(struct inode *, struct page *);
extern void __nfs_read_completion_to_fscache(struct nfs_pgio_header *hdr,
unsigned long bytes);
extern void __nfs_readpage_to_fscache(struct inode *, struct page *);
extern int __nfs_readpage_from_fscache(struct nfs_open_context *,
struct inode *, struct page *);
extern int __nfs_readpages_from_fscache(struct nfs_open_context *,
struct inode *, struct address_space *,
struct list_head *, unsigned *);
extern void __nfs_readpage_to_fscache(struct inode *, struct page *, int);
/*
* wait for a page to complete writing to the cache
*/
static inline void nfs_fscache_wait_on_page_write(struct nfs_inode *nfsi,
struct page *page)
static inline int nfs_fscache_release_page(struct page *page, gfp_t gfp)
{
if (PageFsCache(page))
fscache_wait_on_page_write(nfsi->fscache, page);
}
/*
* release the caching state associated with a page if undergoing complete page
* invalidation
*/
static inline void nfs_fscache_invalidate_page(struct page *page,
struct inode *inode)
{
if (PageFsCache(page))
__nfs_fscache_invalidate_page(page, inode);
if (PageFsCache(page)) {
if (current_is_kswapd() || !(gfp & __GFP_FS))
return false;
wait_on_page_fscache(page);
fscache_note_page_release(nfs_i_fscache(page->mapping->host));
nfs_inc_fscache_stats(page->mapping->host,
NFSIOS_FSCACHE_PAGES_UNCACHED);
}
return true;
}
/*
* Retrieve a page from an inode data storage object.
*/
static inline int nfs_readpage_from_fscache(struct nfs_open_context *ctx,
struct inode *inode,
static inline int nfs_readpage_from_fscache(struct inode *inode,
struct page *page)
{
if (NFS_I(inode)->fscache)
return __nfs_readpage_from_fscache(ctx, inode, page);
return -ENOBUFS;
}
/*
* Retrieve a set of pages from an inode data storage object.
*/
static inline int nfs_readpages_from_fscache(struct nfs_open_context *ctx,
struct inode *inode,
struct address_space *mapping,
struct list_head *pages,
unsigned *nr_pages)
{
if (NFS_I(inode)->fscache)
return __nfs_readpages_from_fscache(ctx, inode, mapping, pages,
nr_pages);
return __nfs_readpage_from_fscache(inode, page);
return -ENOBUFS;
}
@ -156,27 +79,38 @@ static inline int nfs_readpages_from_fscache(struct nfs_open_context *ctx,
* in the cache.
*/
static inline void nfs_readpage_to_fscache(struct inode *inode,
struct page *page,
int sync)
struct page *page)
{
if (PageFsCache(page))
__nfs_readpage_to_fscache(inode, page, sync);
if (NFS_I(inode)->fscache)
__nfs_readpage_to_fscache(inode, page);
}
static inline void nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
struct nfs_inode *nfsi)
{
memset(auxdata, 0, sizeof(*auxdata));
auxdata->mtime_sec = nfsi->vfs_inode.i_mtime.tv_sec;
auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
auxdata->ctime_sec = nfsi->vfs_inode.i_ctime.tv_sec;
auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
}
/*
* Invalidate the contents of fscache for this inode. This will not sleep.
*/
static inline void nfs_fscache_invalidate(struct inode *inode)
static inline void nfs_fscache_invalidate(struct inode *inode, int flags)
{
fscache_invalidate(NFS_I(inode)->fscache);
}
struct nfs_fscache_inode_auxdata auxdata;
struct nfs_inode *nfsi = NFS_I(inode);
/*
* Wait for an object to finish being invalidated.
*/
static inline void nfs_fscache_wait_on_invalidate(struct inode *inode)
{
fscache_wait_on_invalidate(NFS_I(inode)->fscache);
if (nfsi->fscache) {
nfs_fscache_update_auxdata(&auxdata, nfsi);
fscache_invalidate(nfsi->fscache, &auxdata,
i_size_read(&nfsi->vfs_inode), flags);
}
}
/*
@ -190,48 +124,28 @@ static inline const char *nfs_server_fscache_state(struct nfs_server *server)
}
#else /* CONFIG_NFS_FSCACHE */
static inline int nfs_fscache_register(void) { return 0; }
static inline void nfs_fscache_unregister(void) {}
static inline void nfs_fscache_get_client_cookie(struct nfs_client *clp) {}
static inline void nfs_fscache_release_client_cookie(struct nfs_client *clp) {}
static inline void nfs_fscache_release_super_cookie(struct super_block *sb) {}
static inline void nfs_fscache_init_inode(struct inode *inode) {}
static inline void nfs_fscache_clear_inode(struct inode *inode) {}
static inline void nfs_fscache_open_file(struct inode *inode,
struct file *filp) {}
static inline void nfs_fscache_release_file(struct inode *inode, struct file *file) {}
static inline int nfs_fscache_release_page(struct page *page, gfp_t gfp)
{
return 1; /* True: may release page */
}
static inline void nfs_fscache_invalidate_page(struct page *page,
struct inode *inode) {}
static inline void nfs_fscache_wait_on_page_write(struct nfs_inode *nfsi,
struct page *page) {}
static inline int nfs_readpage_from_fscache(struct nfs_open_context *ctx,
struct inode *inode,
static inline int nfs_readpage_from_fscache(struct inode *inode,
struct page *page)
{
return -ENOBUFS;
}
static inline int nfs_readpages_from_fscache(struct nfs_open_context *ctx,
struct inode *inode,
struct address_space *mapping,
struct list_head *pages,
unsigned *nr_pages)
{
return -ENOBUFS;
}
static inline void nfs_readpage_to_fscache(struct inode *inode,
struct page *page, int sync) {}
struct page *page) {}
static inline void nfs_fscache_invalidate(struct inode *inode) {}
static inline void nfs_fscache_wait_on_invalidate(struct inode *inode) {}
static inline void nfs_fscache_invalidate(struct inode *inode, int flags) {}
static inline const char *nfs_server_fscache_state(struct nfs_server *server)
{

11
fs/nfs/inode.c
View File

@ -209,7 +209,7 @@ void nfs_set_cache_invalid(struct inode *inode, unsigned long flags)
if (!nfs_has_xattr_cache(nfsi))
flags &= ~NFS_INO_INVALID_XATTR;
if (flags & NFS_INO_INVALID_DATA)
nfs_fscache_invalidate(inode);
nfs_fscache_invalidate(inode, 0);
flags &= ~(NFS_INO_REVAL_PAGECACHE | NFS_INO_REVAL_FORCED);
nfsi->cache_validity |= flags;
@ -1289,6 +1289,7 @@ static int nfs_invalidate_mapping(struct inode *inode, struct address_space *map
{
int ret;
nfs_fscache_invalidate(inode, 0);
if (mapping->nrpages != 0) {
if (S_ISREG(inode->i_mode)) {
ret = nfs_sync_mapping(mapping);
@ -1300,7 +1301,6 @@ static int nfs_invalidate_mapping(struct inode *inode, struct address_space *map
return ret;
}
nfs_inc_stats(inode, NFSIOS_DATAINVALIDATE);
nfs_fscache_wait_on_invalidate(inode);
dfprintk(PAGECACHE, "NFS: (%s/%Lu) data cache invalidated\n",
inode->i_sb->s_id,
@ -2374,10 +2374,6 @@ static int __init init_nfs_fs(void)
if (err < 0)
goto out9;
err = nfs_fscache_register();
if (err < 0)
goto out8;
err = nfsiod_start();
if (err)
goto out7;
@ -2429,8 +2425,6 @@ static int __init init_nfs_fs(void)
out6:
nfsiod_stop();
out7:
nfs_fscache_unregister();
out8:
unregister_pernet_subsys(&nfs_net_ops);
out9:
nfs_sysfs_exit();
@ -2445,7 +2439,6 @@ static void __exit exit_nfs_fs(void)
nfs_destroy_readpagecache();
nfs_destroy_inodecache();
nfs_destroy_nfspagecache();
nfs_fscache_unregister();
unregister_pernet_subsys(&nfs_net_ops);
rpc_proc_unregister(&init_net, "nfs");
unregister_nfs_fs();

1
fs/nfs/nfstrace.h
View File

@ -42,7 +42,6 @@
{ BIT(NFS_INO_ACL_LRU_SET), "ACL_LRU_SET" }, \
{ BIT(NFS_INO_INVALIDATING), "INVALIDATING" }, \
{ BIT(NFS_INO_FSCACHE), "FSCACHE" }, \
{ BIT(NFS_INO_FSCACHE_LOCK), "FSCACHE_LOCK" }, \
{ BIT(NFS_INO_LAYOUTCOMMIT), "NEED_LAYOUTCOMMIT" }, \
{ BIT(NFS_INO_LAYOUTCOMMITTING), "LAYOUTCOMMIT" }, \
{ BIT(NFS_INO_LAYOUTSTATS), "LAYOUTSTATS" }, \

25
fs/nfs/read.c
View File

@ -123,7 +123,7 @@ static void nfs_readpage_release(struct nfs_page *req, int error)
struct address_space *mapping = page_file_mapping(page);
if (PageUptodate(page))
nfs_readpage_to_fscache(inode, page, 0);
nfs_readpage_to_fscache(inode, page);
else if (!PageError(page) && !PagePrivate(page))
generic_error_remove_page(mapping, page);
unlock_page(page);
@ -305,6 +305,12 @@ readpage_async_filler(void *data, struct page *page)
aligned_len = min_t(unsigned int, ALIGN(len, rsize), PAGE_SIZE);
if (!IS_SYNC(page->mapping->host)) {
error = nfs_readpage_from_fscache(page->mapping->host, page);
if (error == 0)
goto out_unlock;
}
new = nfs_create_request(desc->ctx, page, 0, aligned_len);
if (IS_ERR(new))
goto out_error;
@ -320,6 +326,7 @@ readpage_async_filler(void *data, struct page *page)
return 0;
out_error:
error = PTR_ERR(new);
out_unlock:
unlock_page(page);
out:
return error;
@ -366,12 +373,6 @@ int nfs_readpage(struct file *file, struct page *page)
desc.ctx = get_nfs_open_context(nfs_file_open_context(file));
xchg(&desc.ctx->error, 0);
if (!IS_SYNC(inode)) {
ret = nfs_readpage_from_fscache(desc.ctx, inode, page);
if (ret == 0)
goto out_wait;
}
nfs_pageio_init_read(&desc.pgio, inode, false,
&nfs_async_read_completion_ops);
@ -381,7 +382,6 @@ int nfs_readpage(struct file *file, struct page *page)
nfs_pageio_complete_read(&desc.pgio);
ret = desc.pgio.pg_error < 0 ? desc.pgio.pg_error : 0;
out_wait:
if (!ret) {
ret = wait_on_page_locked_killable(page);
if (!PageUptodate(page) && !ret)
@ -419,14 +419,6 @@ int nfs_readpages(struct file *file, struct address_space *mapping,
} else
desc.ctx = get_nfs_open_context(nfs_file_open_context(file));
/* attempt to read as many of the pages as possible from the cache
* - this returns -ENOBUFS immediately if the cookie is negative
*/
ret = nfs_readpages_from_fscache(desc.ctx, inode, mapping,
pages, &nr_pages);
if (ret == 0)
goto read_complete; /* all pages were read */
nfs_pageio_init_read(&desc.pgio, inode, false,
&nfs_async_read_completion_ops);
@ -434,7 +426,6 @@ int nfs_readpages(struct file *file, struct address_space *mapping,
nfs_pageio_complete_read(&desc.pgio);
read_complete:
put_nfs_open_context(desc.ctx);
out:
trace_nfs_aop_readahead_done(inode, nr_pages, ret);

28
fs/nfs/super.c
View File

@ -1204,42 +1204,42 @@ static int nfs_compare_super(struct super_block *sb, struct fs_context *fc)
}
#ifdef CONFIG_NFS_FSCACHE
static void nfs_get_cache_cookie(struct super_block *sb,
struct nfs_fs_context *ctx)
static int nfs_get_cache_cookie(struct super_block *sb,
struct nfs_fs_context *ctx)
{
struct nfs_server *nfss = NFS_SB(sb);
char *uniq = NULL;
int ulen = 0;
nfss->fscache_key = NULL;
nfss->fscache = NULL;
if (!ctx)
return;
return 0;
if (ctx->clone_data.sb) {
struct nfs_server *mnt_s = NFS_SB(ctx->clone_data.sb);
if (!(mnt_s->options & NFS_OPTION_FSCACHE))
return;
if (mnt_s->fscache_key) {
uniq = mnt_s->fscache_key->key.uniquifier;
ulen = mnt_s->fscache_key->key.uniq_len;
return 0;
if (mnt_s->fscache_uniq) {
uniq = mnt_s->fscache_uniq;
ulen = strlen(uniq);
}
} else {
if (!(ctx->options & NFS_OPTION_FSCACHE))
return;
return 0;
if (ctx->fscache_uniq) {
uniq = ctx->fscache_uniq;
ulen = strlen(ctx->fscache_uniq);
}
}
nfs_fscache_get_super_cookie(sb, uniq, ulen);
return nfs_fscache_get_super_cookie(sb, uniq, ulen);
}
#else
static void nfs_get_cache_cookie(struct super_block *sb,
struct nfs_fs_context *ctx)
static int nfs_get_cache_cookie(struct super_block *sb,
struct nfs_fs_context *ctx)
{
return 0;
}
#endif
@ -1299,7 +1299,9 @@ int nfs_get_tree_common(struct fs_context *fc)
s->s_blocksize_bits = bsize;
s->s_blocksize = 1U << bsize;
}
nfs_get_cache_cookie(s, ctx);
error = nfs_get_cache_cookie(s, ctx);
if (error < 0)
goto error_splat_super;
}
error = nfs_get_root(s, fc);

8
fs/nfs/write.c
View File

@ -294,6 +294,7 @@ static void nfs_grow_file(struct page *page, unsigned int offset, unsigned int c
nfs_inc_stats(inode, NFSIOS_EXTENDWRITE);
out:
spin_unlock(&inode->i_lock);
nfs_fscache_invalidate(inode, 0);
}
/* A writeback failed: mark the page as bad, and invalidate the page cache */
@ -2125,8 +2126,11 @@ int nfs_migrate_page(struct address_space *mapping, struct page *newpage,
if (PagePrivate(page))
return -EBUSY;
if (!nfs_fscache_release_page(page, GFP_KERNEL))
return -EBUSY;
if (PageFsCache(page)) {
if (mode == MIGRATE_ASYNC)
return -EBUSY;
wait_on_page_fscache(page);
}
return migrate_page(mapping, newpage, page, mode);
}

4
include/linux/fs.h
View File

@ -2173,6 +2173,7 @@ struct super_operations {
#define S_ENCRYPTED (1 << 14) /* Encrypted file (using fs/crypto/) */
#define S_CASEFOLD (1 << 15) /* Casefolded file */
#define S_VERITY (1 << 16) /* Verity file (using fs/verity/) */
#define S_KERNEL_FILE (1 << 17) /* File is in use by the kernel (eg. fs/cachefiles) */
/*
* Note that nosuid etc flags are inode-specific: setting some file-system
@ -2342,6 +2343,8 @@ static inline void kiocb_clone(struct kiocb *kiocb, struct kiocb *kiocb_src,
* Used to detect that mark_inode_dirty() should not move
* inode between dirty lists.
*
* I_PINNING_FSCACHE_WB Inode is pinning an fscache object for writeback.
*
* Q: What is the difference between I_WILL_FREE and I_FREEING?
*/
#define I_DIRTY_SYNC (1 << 0)
@ -2364,6 +2367,7 @@ static inline void kiocb_clone(struct kiocb *kiocb, struct kiocb *kiocb_src,
#define I_CREATING (1 << 15)
#define I_DONTCACHE (1 << 16)
#define I_SYNC_QUEUED (1 << 17)
#define I_PINNING_FSCACHE_WB (1 << 18)
#define I_DIRTY_INODE (I_DIRTY_SYNC | I_DIRTY_DATASYNC)
#define I_DIRTY (I_DIRTY_INODE | I_DIRTY_PAGES)

650
include/linux/fscache-cache.h
View File

@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/* General filesystem caching backing cache interface
*
* Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*
* NOTE!!! See:
@ -15,207 +15,34 @@
#define _LINUX_FSCACHE_CACHE_H
#include <linux/fscache.h>
#include <linux/sched.h>
#include <linux/workqueue.h>
#define NR_MAXCACHES BITS_PER_LONG
enum fscache_cache_trace;
enum fscache_cookie_trace;
enum fscache_access_trace;
struct fscache_cache;
struct fscache_cache_ops;
struct fscache_object;
struct fscache_operation;
enum fscache_obj_ref_trace {
fscache_obj_get_add_to_deps,
fscache_obj_get_queue,
fscache_obj_put_alloc_fail,
fscache_obj_put_attach_fail,
fscache_obj_put_drop_obj,
fscache_obj_put_enq_dep,
fscache_obj_put_queue,
fscache_obj_put_work,
fscache_obj_ref__nr_traces
enum fscache_cache_state {
FSCACHE_CACHE_IS_NOT_PRESENT, /* No cache is present for this name */
FSCACHE_CACHE_IS_PREPARING, /* A cache is preparing to come live */
FSCACHE_CACHE_IS_ACTIVE, /* Attached cache is active and can be used */
FSCACHE_CACHE_GOT_IOERROR, /* Attached cache stopped on I/O error */
FSCACHE_CACHE_IS_WITHDRAWN, /* Attached cache is being withdrawn */
#define NR__FSCACHE_CACHE_STATE (FSCACHE_CACHE_IS_WITHDRAWN + 1)
};
/*
* cache tag definition
*/
struct fscache_cache_tag {
struct list_head link;
struct fscache_cache *cache; /* cache referred to by this tag */
unsigned long flags;
#define FSCACHE_TAG_RESERVED 0 /* T if tag is reserved for a cache */
atomic_t usage;
char name[]; /* tag name */
};
/*
* cache definition
* Cache cookie.
*/
struct fscache_cache {
const struct fscache_cache_ops *ops;
struct fscache_cache_tag *tag; /* tag representing this cache */
struct kobject *kobj; /* system representation of this cache */
struct list_head link; /* link in list of caches */
size_t max_index_size; /* maximum size of index data */
char identifier[36]; /* cache label */
/* node management */
struct work_struct op_gc; /* operation garbage collector */
struct list_head object_list; /* list of data/index objects */
struct list_head op_gc_list; /* list of ops to be deleted */
spinlock_t object_list_lock;
spinlock_t op_gc_list_lock;
struct list_head cache_link; /* Link in cache list */
void *cache_priv; /* Private cache data (or NULL) */
refcount_t ref;
atomic_t n_volumes; /* Number of active volumes; */
atomic_t n_accesses; /* Number of in-progress accesses on the cache */
atomic_t object_count; /* no. of live objects in this cache */
struct fscache_object *fsdef; /* object for the fsdef index */
unsigned long flags;
#define FSCACHE_IOERROR 0 /* cache stopped on I/O error */
#define FSCACHE_CACHE_WITHDRAWN 1 /* cache has been withdrawn */
};
extern wait_queue_head_t fscache_cache_cleared_wq;
/*
* operation to be applied to a cache object
* - retrieval initiation operations are done in the context of the process
* that issued them, and not in an async thread pool
*/
typedef void (*fscache_operation_release_t)(struct fscache_operation *op);
typedef void (*fscache_operation_processor_t)(struct fscache_operation *op);
typedef void (*fscache_operation_cancel_t)(struct fscache_operation *op);
enum fscache_operation_state {
FSCACHE_OP_ST_BLANK, /* Op is not yet submitted */
FSCACHE_OP_ST_INITIALISED, /* Op is initialised */
FSCACHE_OP_ST_PENDING, /* Op is blocked from running */
FSCACHE_OP_ST_IN_PROGRESS, /* Op is in progress */
FSCACHE_OP_ST_COMPLETE, /* Op is complete */
FSCACHE_OP_ST_CANCELLED, /* Op has been cancelled */
FSCACHE_OP_ST_DEAD /* Op is now dead */
};
struct fscache_operation {
struct work_struct work; /* record for async ops */
struct list_head pend_link; /* link in object->pending_ops */
struct fscache_object *object; /* object to be operated upon */
unsigned long flags;
#define FSCACHE_OP_TYPE 0x000f /* operation type */
#define FSCACHE_OP_ASYNC 0x0001 /* - async op, processor may sleep for disk */
#define FSCACHE_OP_MYTHREAD 0x0002 /* - processing is done be issuing thread, not pool */
#define FSCACHE_OP_WAITING 4 /* cleared when op is woken */
#define FSCACHE_OP_EXCLUSIVE 5 /* exclusive op, other ops must wait */
#define FSCACHE_OP_DEC_READ_CNT 6 /* decrement object->n_reads on destruction */
#define FSCACHE_OP_UNUSE_COOKIE 7 /* call fscache_unuse_cookie() on completion */
#define FSCACHE_OP_KEEP_FLAGS 0x00f0 /* flags to keep when repurposing an op */
enum fscache_operation_state state;
atomic_t usage;
unsigned debug_id; /* debugging ID */
/* operation processor callback
* - can be NULL if FSCACHE_OP_WAITING is going to be used to perform
* the op in a non-pool thread */
fscache_operation_processor_t processor;
/* Operation cancellation cleanup (optional) */
fscache_operation_cancel_t cancel;
/* operation releaser */
fscache_operation_release_t release;
};
extern atomic_t fscache_op_debug_id;
extern void fscache_op_work_func(struct work_struct *work);
extern void fscache_enqueue_operation(struct fscache_operation *);
extern void fscache_op_complete(struct fscache_operation *, bool);
extern void fscache_put_operation(struct fscache_operation *);
extern void fscache_operation_init(struct fscache_cookie *,
struct fscache_operation *,
fscache_operation_processor_t,
fscache_operation_cancel_t,
fscache_operation_release_t);
/*
* data read operation
*/
struct fscache_retrieval {
struct fscache_operation op;
struct fscache_cookie *cookie; /* The netfs cookie */
struct address_space *mapping; /* netfs pages */
fscache_rw_complete_t end_io_func; /* function to call on I/O completion */
void *context; /* netfs read context (pinned) */
struct list_head to_do; /* list of things to be done by the backend */
atomic_t n_pages; /* number of pages to be retrieved */
};
typedef int (*fscache_page_retrieval_func_t)(struct fscache_retrieval *op,
struct page *page,
gfp_t gfp);
typedef int (*fscache_pages_retrieval_func_t)(struct fscache_retrieval *op,
struct list_head *pages,
unsigned *nr_pages,
gfp_t gfp);
/**
* fscache_get_retrieval - Get an extra reference on a retrieval operation
* @op: The retrieval operation to get a reference on
*
* Get an extra reference on a retrieval operation.
*/
static inline
struct fscache_retrieval *fscache_get_retrieval(struct fscache_retrieval *op)
{
atomic_inc(&op->op.usage);
return op;
}
/**
* fscache_enqueue_retrieval - Enqueue a retrieval operation for processing
* @op: The retrieval operation affected
*
* Enqueue a retrieval operation for processing by the FS-Cache thread pool.
*/
static inline void fscache_enqueue_retrieval(struct fscache_retrieval *op)
{
fscache_enqueue_operation(&op->op);
}
/**
* fscache_retrieval_complete - Record (partial) completion of a retrieval
* @op: The retrieval operation affected
* @n_pages: The number of pages to account for
*/
static inline void fscache_retrieval_complete(struct fscache_retrieval *op,
int n_pages)
{
if (atomic_sub_return_relaxed(n_pages, &op->n_pages) <= 0)
fscache_op_complete(&op->op, false);
}
/**
* fscache_put_retrieval - Drop a reference to a retrieval operation
* @op: The retrieval operation affected
*
* Drop a reference to a retrieval operation.
*/
static inline void fscache_put_retrieval(struct fscache_retrieval *op)
{
fscache_put_operation(&op->op);
}
/*
* cached page storage work item
* - used to do three things:
* - batch writes to the cache
* - do cache writes asynchronously
* - defer writes until cache object lookup completion
*/
struct fscache_storage {
struct fscache_operation op;
pgoff_t store_limit; /* don't write more than this */
unsigned int debug_id;
enum fscache_cache_state state;
char *name;
};
/*
@ -225,341 +52,154 @@ struct fscache_cache_ops {
/* name of cache provider */
const char *name;
/* allocate an object record for a cookie */
struct fscache_object *(*alloc_object)(struct fscache_cache *cache,
struct fscache_cookie *cookie);
/* Acquire a volume */
void (*acquire_volume)(struct fscache_volume *volume);
/* look up the object for a cookie
* - return -ETIMEDOUT to be requeued
*/
int (*lookup_object)(struct fscache_object *object);
/* Free the cache's data attached to a volume */
void (*free_volume)(struct fscache_volume *volume);
/* finished looking up */
void (*lookup_complete)(struct fscache_object *object);
/* Look up a cookie in the cache */
bool (*lookup_cookie)(struct fscache_cookie *cookie);
/* increment the usage count on this object (may fail if unmounting) */
struct fscache_object *(*grab_object)(struct fscache_object *object,
enum fscache_obj_ref_trace why);
/* Withdraw an object without any cookie access counts held */
void (*withdraw_cookie)(struct fscache_cookie *cookie);
/* pin an object in the cache */
int (*pin_object)(struct fscache_object *object);
/* unpin an object in the cache */
void (*unpin_object)(struct fscache_object *object);
/* check the consistency between the backing cache and the FS-Cache
* cookie */
int (*check_consistency)(struct fscache_operation *op);
/* store the updated auxiliary data on an object */
void (*update_object)(struct fscache_object *object);
/* Change the size of a data object */
void (*resize_cookie)(struct netfs_cache_resources *cres,
loff_t new_size);
/* Invalidate an object */
void (*invalidate_object)(struct fscache_operation *op);
bool (*invalidate_cookie)(struct fscache_cookie *cookie);
/* discard the resources pinned by an object and effect retirement if
* necessary */
void (*drop_object)(struct fscache_object *object);
/* Begin an operation for the netfs lib */
bool (*begin_operation)(struct netfs_cache_resources *cres,
enum fscache_want_state want_state);
/* dispose of a reference to an object */
void (*put_object)(struct fscache_object *object,
enum fscache_obj_ref_trace why);
/* sync a cache */
void (*sync_cache)(struct fscache_cache *cache);
/* notification that the attributes of a non-index object (such as
* i_size) have changed */
int (*attr_changed)(struct fscache_object *object);
/* reserve space for an object's data and associated metadata */
int (*reserve_space)(struct fscache_object *object, loff_t i_size);
/* request a backing block for a page be read or allocated in the
* cache */
fscache_page_retrieval_func_t read_or_alloc_page;
/* request backing blocks for a list of pages be read or allocated in
* the cache */
fscache_pages_retrieval_func_t read_or_alloc_pages;
/* request a backing block for a page be allocated in the cache so that
* it can be written directly */
fscache_page_retrieval_func_t allocate_page;
/* request backing blocks for pages be allocated in the cache so that
* they can be written directly */
fscache_pages_retrieval_func_t allocate_pages;
/* write a page to its backing block in the cache */
int (*write_page)(struct fscache_storage *op, struct page *page);
/* detach backing block from a page (optional)
* - must release the cookie lock before returning
* - may sleep
*/
void (*uncache_page)(struct fscache_object *object,
struct page *page);
/* dissociate a cache from all the pages it was backing */
void (*dissociate_pages)(struct fscache_cache *cache);
/* Begin a read operation for the netfs lib */
int (*begin_read_operation)(struct netfs_read_request *rreq,
struct fscache_retrieval *op);
/* Prepare to write to a live cache object */
void (*prepare_to_write)(struct fscache_cookie *cookie);
};
extern struct fscache_cookie fscache_fsdef_index;
/*
* Event list for fscache_object::{event_mask,events}
*/
enum {
FSCACHE_OBJECT_EV_NEW_CHILD, /* T if object has a new child */
FSCACHE_OBJECT_EV_PARENT_READY, /* T if object's parent is ready */
FSCACHE_OBJECT_EV_UPDATE, /* T if object should be updated */
FSCACHE_OBJECT_EV_INVALIDATE, /* T if cache requested object invalidation */
FSCACHE_OBJECT_EV_CLEARED, /* T if accessors all gone */
FSCACHE_OBJECT_EV_ERROR, /* T if fatal error occurred during processing */
FSCACHE_OBJECT_EV_KILL, /* T if netfs relinquished or cache withdrew object */
NR_FSCACHE_OBJECT_EVENTS
};
#define FSCACHE_OBJECT_EVENTS_MASK ((1UL << NR_FSCACHE_OBJECT_EVENTS) - 1)
/*
* States for object state machine.
*/
struct fscache_transition {
unsigned long events;
const struct fscache_state *transit_to;
};
struct fscache_state {
char name[24];
char short_name[8];
const struct fscache_state *(*work)(struct fscache_object *object,
int event);
const struct fscache_transition transitions[];
};
/*
* on-disk cache file or index handle
*/
struct fscache_object {
const struct fscache_state *state; /* Object state machine state */
const struct fscache_transition *oob_table; /* OOB state transition table */
int debug_id; /* debugging ID */
int n_children; /* number of child objects */
int n_ops; /* number of extant ops on object */
int n_obj_ops; /* number of object ops outstanding on object */
int n_in_progress; /* number of ops in progress */
int n_exclusive; /* number of exclusive ops queued or in progress */
atomic_t n_reads; /* number of read ops in progress */
spinlock_t lock; /* state and operations lock */
unsigned long lookup_jif; /* time at which lookup started */
unsigned long oob_event_mask; /* OOB events this object is interested in */
unsigned long event_mask; /* events this object is interested in */
unsigned long events; /* events to be processed by this object
* (order is important - using fls) */
unsigned long flags;
#define FSCACHE_OBJECT_LOCK 0 /* T if object is busy being processed */
#define FSCACHE_OBJECT_PENDING_WRITE 1 /* T if object has pending write */
#define FSCACHE_OBJECT_WAITING 2 /* T if object is waiting on its parent */
#define FSCACHE_OBJECT_IS_LIVE 3 /* T if object is not withdrawn or relinquished */
#define FSCACHE_OBJECT_IS_LOOKED_UP 4 /* T if object has been looked up */
#define FSCACHE_OBJECT_IS_AVAILABLE 5 /* T if object has become active */
#define FSCACHE_OBJECT_RETIRED 6 /* T if object was retired on relinquishment */
#define FSCACHE_OBJECT_KILLED_BY_CACHE 7 /* T if object was killed by the cache */
#define FSCACHE_OBJECT_RUN_AFTER_DEAD 8 /* T if object has been dispatched after death */
struct list_head cache_link; /* link in cache->object_list */
struct hlist_node cookie_link; /* link in cookie->backing_objects */
struct fscache_cache *cache; /* cache that supplied this object */
struct fscache_cookie *cookie; /* netfs's file/index object */
struct fscache_object *parent; /* parent object */
struct work_struct work; /* attention scheduling record */
struct list_head dependents; /* FIFO of dependent objects */
struct list_head dep_link; /* link in parent's dependents list */
struct list_head pending_ops; /* unstarted operations on this object */
pgoff_t store_limit; /* current storage limit */
loff_t store_limit_l; /* current storage limit */
};
extern void fscache_object_init(struct fscache_object *, struct fscache_cookie *,
struct fscache_cache *);
extern void fscache_object_destroy(struct fscache_object *);
extern void fscache_object_lookup_negative(struct fscache_object *object);
extern void fscache_obtained_object(struct fscache_object *object);
static inline bool fscache_object_is_live(struct fscache_object *object)
{
return test_bit(FSCACHE_OBJECT_IS_LIVE, &object->flags);
}
static inline bool fscache_object_is_dying(struct fscache_object *object)
{
return !fscache_object_is_live(object);
}
static inline bool fscache_object_is_available(struct fscache_object *object)
{
return test_bit(FSCACHE_OBJECT_IS_AVAILABLE, &object->flags);
}
static inline bool fscache_cache_is_broken(struct fscache_object *object)
{
return test_bit(FSCACHE_IOERROR, &object->cache->flags);
}
static inline bool fscache_object_is_active(struct fscache_object *object)
{
return fscache_object_is_available(object) &&
fscache_object_is_live(object) &&
!fscache_cache_is_broken(object);
}
/**
* fscache_object_destroyed - Note destruction of an object in a cache
* @cache: The cache from which the object came
*
* Note the destruction and deallocation of an object record in a cache.
*/
static inline void fscache_object_destroyed(struct fscache_cache *cache)
{
if (atomic_dec_and_test(&cache->object_count))
wake_up_all(&fscache_cache_cleared_wq);
}
/**
* fscache_object_lookup_error - Note an object encountered an error
* @object: The object on which the error was encountered
*
* Note that an object encountered a fatal error (usually an I/O error) and
* that it should be withdrawn as soon as possible.
*/
static inline void fscache_object_lookup_error(struct fscache_object *object)
{
set_bit(FSCACHE_OBJECT_EV_ERROR, &object->events);
}
/**
* fscache_set_store_limit - Set the maximum size to be stored in an object
* @object: The object to set the maximum on
* @i_size: The limit to set in bytes
*
* Set the maximum size an object is permitted to reach, implying the highest
* byte that may be written. Intended to be called by the attr_changed() op.
*
* See Documentation/filesystems/caching/backend-api.rst for a complete
* description.
*/
static inline
void fscache_set_store_limit(struct fscache_object *object, loff_t i_size)
{
object->store_limit_l = i_size;
object->store_limit = i_size >> PAGE_SHIFT;
if (i_size & ~PAGE_MASK)
object->store_limit++;
}
/**
* fscache_end_io - End a retrieval operation on a page
* @op: The FS-Cache operation covering the retrieval
* @page: The page that was to be fetched
* @error: The error code (0 if successful)
*
* Note the end of an operation to retrieve a page, as covered by a particular
* operation record.
*/
static inline void fscache_end_io(struct fscache_retrieval *op,
struct page *page, int error)
{
op->end_io_func(page, op->context, error);
}
static inline void __fscache_use_cookie(struct fscache_cookie *cookie)
{
atomic_inc(&cookie->n_active);
}
/**
* fscache_use_cookie - Request usage of cookie attached to an object
* @object: Object description
*
* Request usage of the cookie attached to an object. NULL is returned if the
* relinquishment had reduced the cookie usage count to 0.
*/
static inline bool fscache_use_cookie(struct fscache_object *object)
{
struct fscache_cookie *cookie = object->cookie;
return atomic_inc_not_zero(&cookie->n_active) != 0;
}
static inline bool __fscache_unuse_cookie(struct fscache_cookie *cookie)
{
return atomic_dec_and_test(&cookie->n_active);
}
static inline void __fscache_wake_unused_cookie(struct fscache_cookie *cookie)
{
wake_up_var(&cookie->n_active);
}
/**
* fscache_unuse_cookie - Cease usage of cookie attached to an object
* @object: Object description
*
* Cease usage of the cookie attached to an object. When the users count
* reaches zero then the cookie relinquishment will be permitted to proceed.
*/
static inline void fscache_unuse_cookie(struct fscache_object *object)
{
struct fscache_cookie *cookie = object->cookie;
if (__fscache_unuse_cookie(cookie))
__fscache_wake_unused_cookie(cookie);
}
extern struct workqueue_struct *fscache_wq;
extern wait_queue_head_t fscache_clearance_waiters;
/*
* out-of-line cache backend functions
*/
extern __printf(3, 4)
void fscache_init_cache(struct fscache_cache *cache,
const struct fscache_cache_ops *ops,
const char *idfmt, ...);
extern struct rw_semaphore fscache_addremove_sem;
extern struct fscache_cache *fscache_acquire_cache(const char *name);
extern void fscache_relinquish_cache(struct fscache_cache *cache);
extern int fscache_add_cache(struct fscache_cache *cache,
struct fscache_object *fsdef,
const char *tagname);
const struct fscache_cache_ops *ops,
void *cache_priv);
extern void fscache_withdraw_cache(struct fscache_cache *cache);
extern void fscache_withdraw_volume(struct fscache_volume *volume);
extern void fscache_withdraw_cookie(struct fscache_cookie *cookie);
extern void fscache_io_error(struct fscache_cache *cache);
extern void fscache_mark_page_cached(struct fscache_retrieval *op,
struct page *page);
extern void fscache_end_volume_access(struct fscache_volume *volume,
struct fscache_cookie *cookie,
enum fscache_access_trace why);
extern void fscache_mark_pages_cached(struct fscache_retrieval *op,
struct pagevec *pagevec);
extern struct fscache_cookie *fscache_get_cookie(struct fscache_cookie *cookie,
enum fscache_cookie_trace where);
extern void fscache_put_cookie(struct fscache_cookie *cookie,
enum fscache_cookie_trace where);
extern void fscache_end_cookie_access(struct fscache_cookie *cookie,
enum fscache_access_trace why);
extern void fscache_cookie_lookup_negative(struct fscache_cookie *cookie);
extern void fscache_resume_after_invalidation(struct fscache_cookie *cookie);
extern void fscache_caching_failed(struct fscache_cookie *cookie);
extern bool fscache_wait_for_operation(struct netfs_cache_resources *cred,
enum fscache_want_state state);
extern bool fscache_object_sleep_till_congested(signed long *timeoutp);
/**
* fscache_cookie_state - Read the state of a cookie
* @cookie: The cookie to query
*
* Get the state of a cookie, imposing an ordering between the cookie contents
* and the state value. Paired with fscache_set_cookie_state().
*/
static inline
enum fscache_cookie_state fscache_cookie_state(struct fscache_cookie *cookie)
{
return smp_load_acquire(&cookie->state);
}
extern enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
const void *data,
uint16_t datalen,
loff_t object_size);
/**
* fscache_get_key - Get a pointer to the cookie key
* @cookie: The cookie to query
*
* Return a pointer to the where a cookie's key is stored.
*/
static inline void *fscache_get_key(struct fscache_cookie *cookie)
{
if (cookie->key_len <= sizeof(cookie->inline_key))
return cookie->inline_key;
else
return cookie->key;
}
extern void fscache_object_retrying_stale(struct fscache_object *object);
static inline struct fscache_cookie *fscache_cres_cookie(struct netfs_cache_resources *cres)
{
return cres->cache_priv;
}
enum fscache_why_object_killed {
FSCACHE_OBJECT_IS_STALE,
FSCACHE_OBJECT_NO_SPACE,
FSCACHE_OBJECT_WAS_RETIRED,
FSCACHE_OBJECT_WAS_CULLED,
};
extern void fscache_object_mark_killed(struct fscache_object *object,
enum fscache_why_object_killed why);
/**
* fscache_count_object - Tell fscache that an object has been added
* @cache: The cache to account to
*
* Tell fscache that an object has been added to the cache. This prevents the
* cache from tearing down the cache structure until the object is uncounted.
*/
static inline void fscache_count_object(struct fscache_cache *cache)
{
atomic_inc(&cache->object_count);
}
/**
* fscache_uncount_object - Tell fscache that an object has been removed
* @cache: The cache to account to
*
* Tell fscache that an object has been removed from the cache and will no
* longer be accessed. After this point, the cache cookie may be destroyed.
*/
static inline void fscache_uncount_object(struct fscache_cache *cache)
{
if (atomic_dec_and_test(&cache->object_count))
wake_up_all(&fscache_clearance_waiters);
}
/**
* fscache_wait_for_objects - Wait for all objects to be withdrawn
* @cache: The cache to query
*
* Wait for all extant objects in a cache to finish being withdrawn
* and go away.
*/
static inline void fscache_wait_for_objects(struct fscache_cache *cache)
{
wait_event(fscache_clearance_waiters,
atomic_read(&cache->object_count) == 0);
}
#ifdef CONFIG_FSCACHE_STATS
extern atomic_t fscache_n_read;
extern atomic_t fscache_n_write;
extern atomic_t fscache_n_no_write_space;
extern atomic_t fscache_n_no_create_space;
extern atomic_t fscache_n_culled;
#define fscache_count_read() atomic_inc(&fscache_n_read)
#define fscache_count_write() atomic_inc(&fscache_n_write)
#define fscache_count_no_write_space() atomic_inc(&fscache_n_no_write_space)
#define fscache_count_no_create_space() atomic_inc(&fscache_n_no_create_space)
#define fscache_count_culled() atomic_inc(&fscache_n_culled)
#else
#define fscache_count_read() do {} while(0)
#define fscache_count_write() do {} while(0)
#define fscache_count_no_write_space() do {} while(0)
#define fscache_count_no_create_space() do {} while(0)
#define fscache_count_culled() do {} while(0)
#endif
#endif /* _LINUX_FSCACHE_CACHE_H */

1077
include/linux/fscache.h
View File

File diff suppressed because it is too large Load Diff

15
include/linux/netfs.h
View File

@ -124,6 +124,7 @@ struct netfs_cache_resources {
void *cache_priv;
void *cache_priv2;
unsigned int debug_id; /* Cookie debug ID */
unsigned int inval_counter; /* object->inval_counter at begin_op */
};
/*
@ -195,6 +196,15 @@ struct netfs_read_request_ops {
void (*cleanup)(struct address_space *mapping, void *netfs_priv);
};
/*
* How to handle reading from a hole.
*/
enum netfs_read_from_hole {
NETFS_READ_HOLE_IGNORE,
NETFS_READ_HOLE_CLEAR,
NETFS_READ_HOLE_FAIL,
};
/*
* Table of operations for access to a cache. This is obtained by
* rreq->ops->begin_cache_operation().
@ -207,7 +217,7 @@ struct netfs_cache_ops {
int (*read)(struct netfs_cache_resources *cres,
loff_t start_pos,
struct iov_iter *iter,
bool seek_data,
enum netfs_read_from_hole read_hole,
netfs_io_terminated_t term_func,
void *term_func_priv);
@ -232,7 +242,8 @@ struct netfs_cache_ops {
* actually do.
*/
int (*prepare_write)(struct netfs_cache_resources *cres,
loff_t *_start, size_t *_len, loff_t i_size);
loff_t *_start, size_t *_len, loff_t i_size,
bool no_space_allocated_yet);
};
struct readahead_control;

1
include/linux/nfs_fs.h
View File

@ -275,7 +275,6 @@ struct nfs4_copy_state {
#define NFS_INO_ACL_LRU_SET (2) /* Inode is on the LRU list */
#define NFS_INO_INVALIDATING (3) /* inode is being invalidated */
#define NFS_INO_FSCACHE (5) /* inode can be cached by FS-Cache */
#define NFS_INO_FSCACHE_LOCK (6) /* FS-Cache cookie management lock */
#define NFS_INO_FORCE_READDIR (7) /* force readdirplus */
#define NFS_INO_LAYOUTCOMMIT (9) /* layoutcommit required */
#define NFS_INO_LAYOUTCOMMITTING (10) /* layoutcommit inflight */

9
include/linux/nfs_fs_sb.h
View File

@ -120,11 +120,6 @@ struct nfs_client {
* This is used to generate the mv0 callback address.
*/
char cl_ipaddr[48];
#ifdef CONFIG_NFS_FSCACHE
struct fscache_cookie *fscache; /* client index cache cookie */
#endif
struct net *cl_net;
struct list_head pending_cb_stateids;
};
@ -194,8 +189,8 @@ struct nfs_server {
struct nfs_auth_info auth_info; /* parsed auth flavors */
#ifdef CONFIG_NFS_FSCACHE
struct nfs_fscache_key *fscache_key; /* unique key for superblock */
struct fscache_cookie *fscache; /* superblock cookie */
struct fscache_volume *fscache; /* superblock cookie */
char *fscache_uniq; /* Uniquifier (or NULL) */
#endif
u32 pnfs_blksize; /* layout_blksize attr */

1
include/linux/writeback.h
View File

@ -68,6 +68,7 @@ struct writeback_control {
unsigned for_reclaim:1; /* Invoked from the page allocator */
unsigned range_cyclic:1; /* range_start is cyclic */
unsigned for_sync:1; /* sync(2) WB_SYNC_ALL writeback */
unsigned unpinned_fscache_wb:1; /* Cleared I_PINNING_FSCACHE_WB */
/*
* When writeback IOs are bounced through async layers, only the

529
include/trace/events/cachefiles.h
View File

@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/* CacheFiles tracepoints
*
* Copyright (C) 2016 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#undef TRACE_SYSTEM
@ -19,9 +19,83 @@
#define __CACHEFILES_DECLARE_TRACE_ENUMS_ONCE_ONLY
enum cachefiles_obj_ref_trace {
cachefiles_obj_put_wait_retry = fscache_obj_ref__nr_traces,
cachefiles_obj_put_wait_timeo,
cachefiles_obj_ref__nr_traces
cachefiles_obj_get_ioreq,
cachefiles_obj_new,
cachefiles_obj_put_alloc_fail,
cachefiles_obj_put_detach,
cachefiles_obj_put_ioreq,
cachefiles_obj_see_clean_commit,
cachefiles_obj_see_clean_delete,
cachefiles_obj_see_clean_drop_tmp,
cachefiles_obj_see_lookup_cookie,
cachefiles_obj_see_lookup_failed,
cachefiles_obj_see_withdraw_cookie,
cachefiles_obj_see_withdrawal,
};
enum fscache_why_object_killed {
FSCACHE_OBJECT_IS_STALE,
FSCACHE_OBJECT_IS_WEIRD,
FSCACHE_OBJECT_INVALIDATED,
FSCACHE_OBJECT_NO_SPACE,
FSCACHE_OBJECT_WAS_RETIRED,
FSCACHE_OBJECT_WAS_CULLED,
FSCACHE_VOLUME_IS_WEIRD,
};
enum cachefiles_coherency_trace {
cachefiles_coherency_check_aux,
cachefiles_coherency_check_content,
cachefiles_coherency_check_dirty,
cachefiles_coherency_check_len,
cachefiles_coherency_check_objsize,
cachefiles_coherency_check_ok,
cachefiles_coherency_check_type,
cachefiles_coherency_check_xattr,
cachefiles_coherency_set_fail,
cachefiles_coherency_set_ok,
cachefiles_coherency_vol_check_cmp,
cachefiles_coherency_vol_check_ok,
cachefiles_coherency_vol_check_xattr,
cachefiles_coherency_vol_set_fail,
cachefiles_coherency_vol_set_ok,
};
enum cachefiles_trunc_trace {
cachefiles_trunc_dio_adjust,
cachefiles_trunc_expand_tmpfile,
cachefiles_trunc_shrink,
};
enum cachefiles_prepare_read_trace {
cachefiles_trace_read_after_eof,
cachefiles_trace_read_found_hole,
cachefiles_trace_read_found_part,
cachefiles_trace_read_have_data,
cachefiles_trace_read_no_data,
cachefiles_trace_read_no_file,
cachefiles_trace_read_seek_error,
cachefiles_trace_read_seek_nxio,
};
enum cachefiles_error_trace {
cachefiles_trace_fallocate_error,
cachefiles_trace_getxattr_error,
cachefiles_trace_link_error,
cachefiles_trace_lookup_error,
cachefiles_trace_mkdir_error,
cachefiles_trace_notify_change_error,
cachefiles_trace_open_error,
cachefiles_trace_read_error,
cachefiles_trace_remxattr_error,
cachefiles_trace_rename_error,
cachefiles_trace_seek_error,
cachefiles_trace_setxattr_error,
cachefiles_trace_statfs_error,
cachefiles_trace_tmpfile_error,
cachefiles_trace_trunc_error,
cachefiles_trace_unlink_error,
cachefiles_trace_write_error,
};
#endif
@ -31,21 +105,78 @@ enum cachefiles_obj_ref_trace {
*/
#define cachefiles_obj_kill_traces \
EM(FSCACHE_OBJECT_IS_STALE, "stale") \
EM(FSCACHE_OBJECT_IS_WEIRD, "weird") \
EM(FSCACHE_OBJECT_INVALIDATED, "inval") \
EM(FSCACHE_OBJECT_NO_SPACE, "no_space") \
EM(FSCACHE_OBJECT_WAS_RETIRED, "was_retired") \
E_(FSCACHE_OBJECT_WAS_CULLED, "was_culled")
EM(FSCACHE_OBJECT_WAS_CULLED, "was_culled") \
E_(FSCACHE_VOLUME_IS_WEIRD, "volume_weird")
#define cachefiles_obj_ref_traces \
EM(fscache_obj_get_add_to_deps, "GET add_to_deps") \
EM(fscache_obj_get_queue, "GET queue") \
EM(fscache_obj_put_alloc_fail, "PUT alloc_fail") \
EM(fscache_obj_put_attach_fail, "PUT attach_fail") \
EM(fscache_obj_put_drop_obj, "PUT drop_obj") \
EM(fscache_obj_put_enq_dep, "PUT enq_dep") \
EM(fscache_obj_put_queue, "PUT queue") \
EM(fscache_obj_put_work, "PUT work") \
EM(cachefiles_obj_put_wait_retry, "PUT wait_retry") \
E_(cachefiles_obj_put_wait_timeo, "PUT wait_timeo")
EM(cachefiles_obj_get_ioreq, "GET ioreq") \
EM(cachefiles_obj_new, "NEW obj") \
EM(cachefiles_obj_put_alloc_fail, "PUT alloc_fail") \
EM(cachefiles_obj_put_detach, "PUT detach") \
EM(cachefiles_obj_put_ioreq, "PUT ioreq") \
EM(cachefiles_obj_see_clean_commit, "SEE clean_commit") \
EM(cachefiles_obj_see_clean_delete, "SEE clean_delete") \
EM(cachefiles_obj_see_clean_drop_tmp, "SEE clean_drop_tmp") \
EM(cachefiles_obj_see_lookup_cookie, "SEE lookup_cookie") \
EM(cachefiles_obj_see_lookup_failed, "SEE lookup_failed") \
EM(cachefiles_obj_see_withdraw_cookie, "SEE withdraw_cookie") \
E_(cachefiles_obj_see_withdrawal, "SEE withdrawal")
#define cachefiles_coherency_traces \
EM(cachefiles_coherency_check_aux, "BAD aux ") \
EM(cachefiles_coherency_check_content, "BAD cont") \
EM(cachefiles_coherency_check_dirty, "BAD dirt") \
EM(cachefiles_coherency_check_len, "BAD len ") \
EM(cachefiles_coherency_check_objsize, "BAD osiz") \
EM(cachefiles_coherency_check_ok, "OK ") \
EM(cachefiles_coherency_check_type, "BAD type") \
EM(cachefiles_coherency_check_xattr, "BAD xatt") \
EM(cachefiles_coherency_set_fail, "SET fail") \
EM(cachefiles_coherency_set_ok, "SET ok ") \
EM(cachefiles_coherency_vol_check_cmp, "VOL BAD cmp ") \
EM(cachefiles_coherency_vol_check_ok, "VOL OK ") \
EM(cachefiles_coherency_vol_check_xattr,"VOL BAD xatt") \
EM(cachefiles_coherency_vol_set_fail, "VOL SET fail") \
E_(cachefiles_coherency_vol_set_ok, "VOL SET ok ")
#define cachefiles_trunc_traces \
EM(cachefiles_trunc_dio_adjust, "DIOADJ") \
EM(cachefiles_trunc_expand_tmpfile, "EXPTMP") \
E_(cachefiles_trunc_shrink, "SHRINK")
#define cachefiles_prepare_read_traces \
EM(cachefiles_trace_read_after_eof, "after-eof ") \
EM(cachefiles_trace_read_found_hole, "found-hole") \
EM(cachefiles_trace_read_found_part, "found-part") \
EM(cachefiles_trace_read_have_data, "have-data ") \
EM(cachefiles_trace_read_no_data, "no-data ") \
EM(cachefiles_trace_read_no_file, "no-file ") \
EM(cachefiles_trace_read_seek_error, "seek-error") \
E_(cachefiles_trace_read_seek_nxio, "seek-enxio")
#define cachefiles_error_traces \
EM(cachefiles_trace_fallocate_error, "fallocate") \
EM(cachefiles_trace_getxattr_error, "getxattr") \
EM(cachefiles_trace_link_error, "link") \
EM(cachefiles_trace_lookup_error, "lookup") \
EM(cachefiles_trace_mkdir_error, "mkdir") \
EM(cachefiles_trace_notify_change_error, "notify_change") \
EM(cachefiles_trace_open_error, "open") \
EM(cachefiles_trace_read_error, "read") \
EM(cachefiles_trace_remxattr_error, "remxattr") \
EM(cachefiles_trace_rename_error, "rename") \
EM(cachefiles_trace_seek_error, "seek") \
EM(cachefiles_trace_setxattr_error, "setxattr") \
EM(cachefiles_trace_statfs_error, "statfs") \
EM(cachefiles_trace_tmpfile_error, "tmpfile") \
EM(cachefiles_trace_trunc_error, "trunc") \
EM(cachefiles_trace_unlink_error, "unlink") \
E_(cachefiles_trace_write_error, "write")
/*
* Export enum symbols via userspace.
@ -57,6 +188,10 @@ enum cachefiles_obj_ref_trace {
cachefiles_obj_kill_traces;
cachefiles_obj_ref_traces;
cachefiles_coherency_traces;
cachefiles_trunc_traces;
cachefiles_prepare_read_traces;
cachefiles_error_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@ -69,12 +204,12 @@ cachefiles_obj_ref_traces;
TRACE_EVENT(cachefiles_ref,
TP_PROTO(struct cachefiles_object *obj,
struct fscache_cookie *cookie,
enum cachefiles_obj_ref_trace why,
int usage),
TP_PROTO(unsigned int object_debug_id,
unsigned int cookie_debug_id,
int usage,
enum cachefiles_obj_ref_trace why),
TP_ARGS(obj, cookie, why, usage),
TP_ARGS(object_debug_id, cookie_debug_id, usage, why),
/* Note that obj may be NULL */
TP_STRUCT__entry(
@ -85,8 +220,8 @@ TRACE_EVENT(cachefiles_ref,
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->cookie = cookie->debug_id;
__entry->obj = object_debug_id;
__entry->cookie = cookie_debug_id;
__entry->usage = usage;
__entry->why = why;
),
@ -98,69 +233,65 @@ TRACE_EVENT(cachefiles_ref,
TRACE_EVENT(cachefiles_lookup,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de,
struct inode *inode),
struct dentry *de),
TP_ARGS(obj, de, inode),
TP_ARGS(obj, de),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(struct dentry *, de )
__field(struct inode *, inode )
__field(short, error )
__field(unsigned long, ino )
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->inode = inode;
__entry->obj = obj->debug_id;
__entry->ino = (!IS_ERR(de) && d_backing_inode(de) ?
d_backing_inode(de)->i_ino : 0);
__entry->error = IS_ERR(de) ? PTR_ERR(de) : 0;
),
TP_printk("o=%08x d=%p i=%p",
__entry->obj, __entry->de, __entry->inode)
TP_printk("o=%08x i=%lx e=%d",
__entry->obj, __entry->ino, __entry->error)
);
TRACE_EVENT(cachefiles_mkdir,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de, int ret),
TRACE_EVENT(cachefiles_tmpfile,
TP_PROTO(struct cachefiles_object *obj, struct inode *backer),
TP_ARGS(obj, de, ret),
TP_ARGS(obj, backer),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(struct dentry *, de )
__field(int, ret )
__field(unsigned int, obj )
__field(unsigned int, backer )
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->ret = ret;
__entry->obj = obj->debug_id;
__entry->backer = backer->i_ino;
),
TP_printk("o=%08x d=%p r=%u",
__entry->obj, __entry->de, __entry->ret)
TP_printk("o=%08x b=%08x",
__entry->obj,
__entry->backer)
);
TRACE_EVENT(cachefiles_create,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de, int ret),
TRACE_EVENT(cachefiles_link,
TP_PROTO(struct cachefiles_object *obj, struct inode *backer),
TP_ARGS(obj, de, ret),
TP_ARGS(obj, backer),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(struct dentry *, de )
__field(int, ret )
__field(unsigned int, obj )
__field(unsigned int, backer )
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->ret = ret;
__entry->obj = obj->debug_id;
__entry->backer = backer->i_ino;
),
TP_printk("o=%08x d=%p r=%u",
__entry->obj, __entry->de, __entry->ret)
TP_printk("o=%08x b=%08x",
__entry->obj,
__entry->backer)
);
TRACE_EVENT(cachefiles_unlink,
@ -178,7 +309,7 @@ TRACE_EVENT(cachefiles_unlink,
),
TP_fast_assign(
__entry->obj = obj ? obj->fscache.debug_id : UINT_MAX;
__entry->obj = obj ? obj->debug_id : UINT_MAX;
__entry->de = de;
__entry->why = why;
),
@ -205,7 +336,7 @@ TRACE_EVENT(cachefiles_rename,
),
TP_fast_assign(
__entry->obj = obj ? obj->fscache.debug_id : UINT_MAX;
__entry->obj = obj ? obj->debug_id : UINT_MAX;
__entry->de = de;
__entry->to = to;
__entry->why = why;
@ -216,103 +347,285 @@ TRACE_EVENT(cachefiles_rename,
__print_symbolic(__entry->why, cachefiles_obj_kill_traces))
);
TRACE_EVENT(cachefiles_mark_active,
TRACE_EVENT(cachefiles_coherency,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de),
ino_t ino,
enum cachefiles_content content,
enum cachefiles_coherency_trace why),
TP_ARGS(obj, de),
TP_ARGS(obj, ino, content, why),
/* Note that obj may be NULL */
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(struct dentry *, de )
__field(unsigned int, obj )
__field(enum cachefiles_coherency_trace, why )
__field(enum cachefiles_content, content )
__field(u64, ino )
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->obj = obj->debug_id;
__entry->why = why;
__entry->content = content;
__entry->ino = ino;
),
TP_printk("o=%08x d=%p",
__entry->obj, __entry->de)
TP_printk("o=%08x %s i=%llx c=%u",
__entry->obj,
__print_symbolic(__entry->why, cachefiles_coherency_traces),
__entry->ino,
__entry->content)
);
TRACE_EVENT(cachefiles_wait_active,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de,
struct cachefiles_object *xobj),
TRACE_EVENT(cachefiles_vol_coherency,
TP_PROTO(struct cachefiles_volume *volume,
ino_t ino,
enum cachefiles_coherency_trace why),
TP_ARGS(obj, de, xobj),
TP_ARGS(volume, ino, why),
/* Note that obj may be NULL */
TP_STRUCT__entry(
__field(unsigned int, vol )
__field(enum cachefiles_coherency_trace, why )
__field(u64, ino )
),
TP_fast_assign(
__entry->vol = volume->vcookie->debug_id;
__entry->why = why;
__entry->ino = ino;
),
TP_printk("V=%08x %s i=%llx",
__entry->vol,
__print_symbolic(__entry->why, cachefiles_coherency_traces),
__entry->ino)
);
TRACE_EVENT(cachefiles_prep_read,
TP_PROTO(struct netfs_read_subrequest *sreq,
enum netfs_read_source source,
enum cachefiles_prepare_read_trace why,
ino_t cache_inode),
TP_ARGS(sreq, source, why, cache_inode),
TP_STRUCT__entry(
__field(unsigned int, rreq )
__field(unsigned short, index )
__field(unsigned short, flags )
__field(enum netfs_read_source, source )
__field(enum cachefiles_prepare_read_trace, why )
__field(size_t, len )
__field(loff_t, start )
__field(unsigned int, netfs_inode )
__field(unsigned int, cache_inode )
),
TP_fast_assign(
__entry->rreq = sreq->rreq->debug_id;
__entry->index = sreq->debug_index;
__entry->flags = sreq->flags;
__entry->source = source;
__entry->why = why;
__entry->len = sreq->len;
__entry->start = sreq->start;
__entry->netfs_inode = sreq->rreq->inode->i_ino;
__entry->cache_inode = cache_inode;
),
TP_printk("R=%08x[%u] %s %s f=%02x s=%llx %zx ni=%x b=%x",
__entry->rreq, __entry->index,
__print_symbolic(__entry->source, netfs_sreq_sources),
__print_symbolic(__entry->why, cachefiles_prepare_read_traces),
__entry->flags,
__entry->start, __entry->len,
__entry->netfs_inode, __entry->cache_inode)
);
TRACE_EVENT(cachefiles_read,
TP_PROTO(struct cachefiles_object *obj,
struct inode *backer,
loff_t start,
size_t len),
TP_ARGS(obj, backer, start, len),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(unsigned int, backer )
__field(size_t, len )
__field(loff_t, start )
),
TP_fast_assign(
__entry->obj = obj->debug_id;
__entry->backer = backer->i_ino;
__entry->start = start;
__entry->len = len;
),
TP_printk("o=%08x b=%08x s=%llx l=%zx",
__entry->obj,
__entry->backer,
__entry->start,
__entry->len)
);
TRACE_EVENT(cachefiles_write,
TP_PROTO(struct cachefiles_object *obj,
struct inode *backer,
loff_t start,
size_t len),
TP_ARGS(obj, backer, start, len),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(unsigned int, backer )
__field(size_t, len )
__field(loff_t, start )
),
TP_fast_assign(
__entry->obj = obj->debug_id;
__entry->backer = backer->i_ino;
__entry->start = start;
__entry->len = len;
),
TP_printk("o=%08x b=%08x s=%llx l=%zx",
__entry->obj,
__entry->backer,
__entry->start,
__entry->len)
);
TRACE_EVENT(cachefiles_trunc,
TP_PROTO(struct cachefiles_object *obj, struct inode *backer,
loff_t from, loff_t to, enum cachefiles_trunc_trace why),
TP_ARGS(obj, backer, from, to, why),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(unsigned int, backer )
__field(enum cachefiles_trunc_trace, why )
__field(loff_t, from )
__field(loff_t, to )
),
TP_fast_assign(
__entry->obj = obj->debug_id;
__entry->backer = backer->i_ino;
__entry->from = from;
__entry->to = to;
__entry->why = why;
),
TP_printk("o=%08x b=%08x %s l=%llx->%llx",
__entry->obj,
__entry->backer,
__print_symbolic(__entry->why, cachefiles_trunc_traces),
__entry->from,
__entry->to)
);
TRACE_EVENT(cachefiles_mark_active,
TP_PROTO(struct cachefiles_object *obj,
struct inode *inode),
TP_ARGS(obj, inode),
/* Note that obj may be NULL */
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(unsigned int, xobj )
__field(struct dentry *, de )
__field(u16, flags )
__field(u16, fsc_flags )
__field(ino_t, inode )
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->xobj = xobj->fscache.debug_id;
__entry->flags = xobj->flags;
__entry->fsc_flags = xobj->fscache.flags;
__entry->obj = obj ? obj->debug_id : 0;
__entry->inode = inode->i_ino;
),
TP_printk("o=%08x d=%p wo=%08x wf=%x wff=%x",
__entry->obj, __entry->de, __entry->xobj,
__entry->flags, __entry->fsc_flags)
TP_printk("o=%08x i=%lx",
__entry->obj, __entry->inode)
);
TRACE_EVENT(cachefiles_mark_inactive,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de,
struct inode *inode),
TP_ARGS(obj, de, inode),
TP_ARGS(obj, inode),
/* Note that obj may be NULL */
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(struct dentry *, de )
__field(struct inode *, inode )
__field(ino_t, inode )
),
TP_fast_assign(
__entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->inode = inode;
__entry->obj = obj ? obj->debug_id : 0;
__entry->inode = inode->i_ino;
),
TP_printk("o=%08x d=%p i=%p",
__entry->obj, __entry->de, __entry->inode)
TP_printk("o=%08x i=%lx",
__entry->obj, __entry->inode)
);
TRACE_EVENT(cachefiles_mark_buried,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de,
enum fscache_why_object_killed why),
TRACE_EVENT(cachefiles_vfs_error,
TP_PROTO(struct cachefiles_object *obj, struct inode *backer,
int error, enum cachefiles_error_trace where),
TP_ARGS(obj, de, why),
TP_ARGS(obj, backer, error, where),
/* Note that obj may be NULL */
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(struct dentry *, de )
__field(enum fscache_why_object_killed, why )
__field(unsigned int, obj )
__field(unsigned int, backer )
__field(enum cachefiles_error_trace, where )
__field(short, error )
),
TP_fast_assign(
__entry->obj = obj ? obj->fscache.debug_id : UINT_MAX;
__entry->de = de;
__entry->why = why;
__entry->obj = obj ? obj->debug_id : 0;
__entry->backer = backer->i_ino;
__entry->error = error;
__entry->where = where;
),
TP_printk("o=%08x d=%p w=%s",
__entry->obj, __entry->de,
__print_symbolic(__entry->why, cachefiles_obj_kill_traces))
TP_printk("o=%08x b=%08x %s e=%d",
__entry->obj,
__entry->backer,
__print_symbolic(__entry->where, cachefiles_error_traces),
__entry->error)
);
TRACE_EVENT(cachefiles_io_error,
TP_PROTO(struct cachefiles_object *obj, struct inode *backer,
int error, enum cachefiles_error_trace where),
TP_ARGS(obj, backer, error, where),
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(unsigned int, backer )
__field(enum cachefiles_error_trace, where )
__field(short, error )
),
TP_fast_assign(
__entry->obj = obj ? obj->debug_id : 0;
__entry->backer = backer->i_ino;
__entry->error = error;
__entry->where = where;
),
TP_printk("o=%08x b=%08x %s e=%d",
__entry->obj,
__entry->backer,
__print_symbolic(__entry->where, cachefiles_error_traces),
__entry->error)
);
#endif /* _TRACE_CACHEFILES_H */

706
include/trace/events/fscache.h
View File

@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/* FS-Cache tracepoints
*
* Copyright (C) 2016 Red Hat, Inc. All Rights Reserved.
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#undef TRACE_SYSTEM
@ -19,65 +19,83 @@
#ifndef __FSCACHE_DECLARE_TRACE_ENUMS_ONCE_ONLY
#define __FSCACHE_DECLARE_TRACE_ENUMS_ONCE_ONLY
enum fscache_cache_trace {
fscache_cache_collision,
fscache_cache_get_acquire,
fscache_cache_new_acquire,
fscache_cache_put_alloc_volume,
fscache_cache_put_cache,
fscache_cache_put_prep_failed,
fscache_cache_put_relinquish,
fscache_cache_put_volume,
};
enum fscache_volume_trace {
fscache_volume_collision,
fscache_volume_get_cookie,
fscache_volume_get_create_work,
fscache_volume_get_hash_collision,
fscache_volume_free,
fscache_volume_new_acquire,
fscache_volume_put_cookie,
fscache_volume_put_create_work,
fscache_volume_put_hash_collision,
fscache_volume_put_relinquish,
fscache_volume_see_create_work,
fscache_volume_see_hash_wake,
fscache_volume_wait_create_work,
};
enum fscache_cookie_trace {
fscache_cookie_collision,
fscache_cookie_discard,
fscache_cookie_get_acquire_parent,
fscache_cookie_get_attach_object,
fscache_cookie_get_reacquire,
fscache_cookie_get_register_netfs,
fscache_cookie_put_acquire_nobufs,
fscache_cookie_put_dup_netfs,
fscache_cookie_put_relinquish,
fscache_cookie_get_end_access,
fscache_cookie_get_hash_collision,
fscache_cookie_get_inval_work,
fscache_cookie_get_lru,
fscache_cookie_get_use_work,
fscache_cookie_new_acquire,
fscache_cookie_put_hash_collision,
fscache_cookie_put_lru,
fscache_cookie_put_object,
fscache_cookie_put_parent,
fscache_cookie_put_over_queued,
fscache_cookie_put_relinquish,
fscache_cookie_put_withdrawn,
fscache_cookie_put_work,
fscache_cookie_see_active,
fscache_cookie_see_lru_discard,
fscache_cookie_see_lru_do_one,
fscache_cookie_see_relinquish,
fscache_cookie_see_withdraw,
fscache_cookie_see_work,
};
enum fscache_page_trace {
fscache_page_cached,
fscache_page_inval,
fscache_page_maybe_release,
fscache_page_radix_clear_store,
fscache_page_radix_delete,
fscache_page_radix_insert,
fscache_page_radix_pend2store,
fscache_page_radix_set_pend,
fscache_page_uncache,
fscache_page_write,
fscache_page_write_end,
fscache_page_write_end_pend,
fscache_page_write_end_noc,
fscache_page_write_wait,
fscache_page_trace__nr
enum fscache_active_trace {
fscache_active_use,
fscache_active_use_modify,
fscache_active_unuse,
};
enum fscache_op_trace {
fscache_op_cancel,
fscache_op_cancel_all,
fscache_op_cancelled,
fscache_op_completed,
fscache_op_enqueue_async,
fscache_op_enqueue_mythread,
fscache_op_gc,
fscache_op_init,
fscache_op_put,
fscache_op_run,
fscache_op_signal,
fscache_op_submit,
fscache_op_submit_ex,
fscache_op_work,
fscache_op_trace__nr
};
enum fscache_page_op_trace {
fscache_page_op_alloc_one,
fscache_page_op_attr_changed,
fscache_page_op_check_consistency,
fscache_page_op_invalidate,
fscache_page_op_retr_multi,
fscache_page_op_retr_one,
fscache_page_op_write_one,
fscache_page_op_trace__nr
enum fscache_access_trace {
fscache_access_acquire_volume,
fscache_access_acquire_volume_end,
fscache_access_cache_pin,
fscache_access_cache_unpin,
fscache_access_invalidate_cookie,
fscache_access_invalidate_cookie_end,
fscache_access_io_end,
fscache_access_io_not_live,
fscache_access_io_read,
fscache_access_io_resize,
fscache_access_io_wait,
fscache_access_io_write,
fscache_access_lookup_cookie,
fscache_access_lookup_cookie_end,
fscache_access_lookup_cookie_end_failed,
fscache_access_relinquish_volume,
fscache_access_relinquish_volume_end,
fscache_access_unlive,
};
#endif
@ -85,59 +103,79 @@ enum fscache_page_op_trace {
/*
* Declare tracing information enums and their string mappings for display.
*/
#define fscache_cache_traces \
EM(fscache_cache_collision, "*COLLIDE*") \
EM(fscache_cache_get_acquire, "GET acq ") \
EM(fscache_cache_new_acquire, "NEW acq ") \
EM(fscache_cache_put_alloc_volume, "PUT alvol") \
EM(fscache_cache_put_cache, "PUT cache") \
EM(fscache_cache_put_prep_failed, "PUT pfail") \
EM(fscache_cache_put_relinquish, "PUT relnq") \
E_(fscache_cache_put_volume, "PUT vol ")
#define fscache_volume_traces \
EM(fscache_volume_collision, "*COLLIDE*") \
EM(fscache_volume_get_cookie, "GET cook ") \
EM(fscache_volume_get_create_work, "GET creat") \
EM(fscache_volume_get_hash_collision, "GET hcoll") \
EM(fscache_volume_free, "FREE ") \
EM(fscache_volume_new_acquire, "NEW acq ") \
EM(fscache_volume_put_cookie, "PUT cook ") \
EM(fscache_volume_put_create_work, "PUT creat") \
EM(fscache_volume_put_hash_collision, "PUT hcoll") \
EM(fscache_volume_put_relinquish, "PUT relnq") \
EM(fscache_volume_see_create_work, "SEE creat") \
EM(fscache_volume_see_hash_wake, "SEE hwake") \
E_(fscache_volume_wait_create_work, "WAIT crea")
#define fscache_cookie_traces \
EM(fscache_cookie_collision, "*COLLISION*") \
EM(fscache_cookie_discard, "DISCARD") \
EM(fscache_cookie_get_acquire_parent, "GET prn") \
EM(fscache_cookie_get_attach_object, "GET obj") \
EM(fscache_cookie_get_reacquire, "GET raq") \
EM(fscache_cookie_get_register_netfs, "GET net") \
EM(fscache_cookie_put_acquire_nobufs, "PUT nbf") \
EM(fscache_cookie_put_dup_netfs, "PUT dnt") \
EM(fscache_cookie_put_relinquish, "PUT rlq") \
EM(fscache_cookie_put_object, "PUT obj") \
E_(fscache_cookie_put_parent, "PUT prn")
EM(fscache_cookie_collision, "*COLLIDE*") \
EM(fscache_cookie_discard, "DISCARD ") \
EM(fscache_cookie_get_attach_object, "GET attch") \
EM(fscache_cookie_get_hash_collision, "GET hcoll") \
EM(fscache_cookie_get_end_access, "GQ endac") \
EM(fscache_cookie_get_inval_work, "GQ inval") \
EM(fscache_cookie_get_lru, "GET lru ") \
EM(fscache_cookie_get_use_work, "GQ use ") \
EM(fscache_cookie_new_acquire, "NEW acq ") \
EM(fscache_cookie_put_hash_collision, "PUT hcoll") \
EM(fscache_cookie_put_lru, "PUT lru ") \
EM(fscache_cookie_put_object, "PUT obj ") \
EM(fscache_cookie_put_over_queued, "PQ overq") \
EM(fscache_cookie_put_relinquish, "PUT relnq") \
EM(fscache_cookie_put_withdrawn, "PUT wthdn") \
EM(fscache_cookie_put_work, "PQ work ") \
EM(fscache_cookie_see_active, "- activ") \
EM(fscache_cookie_see_lru_discard, "- x-lru") \
EM(fscache_cookie_see_lru_do_one, "- lrudo") \
EM(fscache_cookie_see_relinquish, "- x-rlq") \
EM(fscache_cookie_see_withdraw, "- x-wth") \
E_(fscache_cookie_see_work, "- work ")
#define fscache_page_traces \
EM(fscache_page_cached, "Cached ") \
EM(fscache_page_inval, "InvalPg") \
EM(fscache_page_maybe_release, "MayRels") \
EM(fscache_page_uncache, "Uncache") \
EM(fscache_page_radix_clear_store, "RxCStr ") \
EM(fscache_page_radix_delete, "RxDel ") \
EM(fscache_page_radix_insert, "RxIns ") \
EM(fscache_page_radix_pend2store, "RxP2S ") \
EM(fscache_page_radix_set_pend, "RxSPend ") \
EM(fscache_page_write, "WritePg") \
EM(fscache_page_write_end, "EndPgWr") \
EM(fscache_page_write_end_pend, "EndPgWP") \
EM(fscache_page_write_end_noc, "EndPgNC") \
E_(fscache_page_write_wait, "WtOnWrt")
#define fscache_active_traces \
EM(fscache_active_use, "USE ") \
EM(fscache_active_use_modify, "USE-m ") \
E_(fscache_active_unuse, "UNUSE ")
#define fscache_op_traces \
EM(fscache_op_cancel, "Cancel1") \
EM(fscache_op_cancel_all, "CancelA") \
EM(fscache_op_cancelled, "Canclld") \
EM(fscache_op_completed, "Complet") \
EM(fscache_op_enqueue_async, "EnqAsyn") \
EM(fscache_op_enqueue_mythread, "EnqMyTh") \
EM(fscache_op_gc, "GC ") \
EM(fscache_op_init, "Init ") \
EM(fscache_op_put, "Put ") \
EM(fscache_op_run, "Run ") \
EM(fscache_op_signal, "Signal ") \
EM(fscache_op_submit, "Submit ") \
EM(fscache_op_submit_ex, "SubmitX") \
E_(fscache_op_work, "Work ")
#define fscache_page_op_traces \
EM(fscache_page_op_alloc_one, "Alloc1 ") \
EM(fscache_page_op_attr_changed, "AttrChg") \
EM(fscache_page_op_check_consistency, "CheckCn") \
EM(fscache_page_op_invalidate, "Inval ") \
EM(fscache_page_op_retr_multi, "RetrMul") \
EM(fscache_page_op_retr_one, "Retr1 ") \
E_(fscache_page_op_write_one, "Write1 ")
#define fscache_access_traces \
EM(fscache_access_acquire_volume, "BEGIN acq_vol") \
EM(fscache_access_acquire_volume_end, "END acq_vol") \
EM(fscache_access_cache_pin, "PIN cache ") \
EM(fscache_access_cache_unpin, "UNPIN cache ") \
EM(fscache_access_invalidate_cookie, "BEGIN inval ") \
EM(fscache_access_invalidate_cookie_end,"END inval ") \
EM(fscache_access_io_end, "END io ") \
EM(fscache_access_io_not_live, "END io_notl") \
EM(fscache_access_io_read, "BEGIN io_read") \
EM(fscache_access_io_resize, "BEGIN io_resz") \
EM(fscache_access_io_wait, "WAIT io ") \
EM(fscache_access_io_write, "BEGIN io_writ") \
EM(fscache_access_lookup_cookie, "BEGIN lookup ") \
EM(fscache_access_lookup_cookie_end, "END lookup ") \
EM(fscache_access_lookup_cookie_end_failed,"END lookupf") \
EM(fscache_access_relinquish_volume, "BEGIN rlq_vol") \
EM(fscache_access_relinquish_volume_end,"END rlq_vol") \
E_(fscache_access_unlive, "END unlive ")
/*
* Export enum symbols via userspace.
@ -147,7 +185,10 @@ enum fscache_page_op_trace {
#define EM(a, b) TRACE_DEFINE_ENUM(a);
#define E_(a, b) TRACE_DEFINE_ENUM(a);
fscache_cache_traces;
fscache_volume_traces;
fscache_cookie_traces;
fscache_access_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@ -159,6 +200,56 @@ fscache_cookie_traces;
#define E_(a, b) { a, b }
TRACE_EVENT(fscache_cache,
TP_PROTO(unsigned int cache_debug_id,
int usage,
enum fscache_cache_trace where),
TP_ARGS(cache_debug_id, usage, where),
TP_STRUCT__entry(
__field(unsigned int, cache )
__field(int, usage )
__field(enum fscache_cache_trace, where )
),
TP_fast_assign(
__entry->cache = cache_debug_id;
__entry->usage = usage;
__entry->where = where;
),
TP_printk("C=%08x %s r=%d",
__entry->cache,
__print_symbolic(__entry->where, fscache_cache_traces),
__entry->usage)
);
TRACE_EVENT(fscache_volume,
TP_PROTO(unsigned int volume_debug_id,
int usage,
enum fscache_volume_trace where),
TP_ARGS(volume_debug_id, usage, where),
TP_STRUCT__entry(
__field(unsigned int, volume )
__field(int, usage )
__field(enum fscache_volume_trace, where )
),
TP_fast_assign(
__entry->volume = volume_debug_id;
__entry->usage = usage;
__entry->where = where;
),
TP_printk("V=%08x %s u=%d",
__entry->volume,
__print_symbolic(__entry->where, fscache_volume_traces),
__entry->usage)
);
TRACE_EVENT(fscache_cookie,
TP_PROTO(unsigned int cookie_debug_id,
int ref,
@ -168,39 +259,144 @@ TRACE_EVENT(fscache_cookie,
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(enum fscache_cookie_trace, where )
__field(int, ref )
__field(enum fscache_cookie_trace, where )
),
TP_fast_assign(
__entry->cookie = cookie_debug_id;
__entry->where = where;
__entry->ref = ref;
__entry->where = where;
),
TP_printk("%s c=%08x r=%d",
TP_printk("c=%08x %s r=%d",
__entry->cookie,
__print_symbolic(__entry->where, fscache_cookie_traces),
__entry->cookie, __entry->ref)
__entry->ref)
);
TRACE_EVENT(fscache_netfs,
TP_PROTO(struct fscache_netfs *netfs),
TRACE_EVENT(fscache_active,
TP_PROTO(unsigned int cookie_debug_id,
int ref,
int n_active,
int n_accesses,
enum fscache_active_trace why),
TP_ARGS(netfs),
TP_ARGS(cookie_debug_id, ref, n_active, n_accesses, why),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__array(char, name, 8 )
__field(int, ref )
__field(int, n_active )
__field(int, n_accesses )
__field(enum fscache_active_trace, why )
),
TP_fast_assign(
__entry->cookie = netfs->primary_index->debug_id;
strncpy(__entry->name, netfs->name, 8);
__entry->name[7] = 0;
__entry->cookie = cookie_debug_id;
__entry->ref = ref;
__entry->n_active = n_active;
__entry->n_accesses = n_accesses;
__entry->why = why;
),
TP_printk("c=%08x n=%s",
__entry->cookie, __entry->name)
TP_printk("c=%08x %s r=%d a=%d c=%d",
__entry->cookie,
__print_symbolic(__entry->why, fscache_active_traces),
__entry->ref,
__entry->n_accesses,
__entry->n_active)
);
TRACE_EVENT(fscache_access_cache,
TP_PROTO(unsigned int cache_debug_id,
int ref,
int n_accesses,
enum fscache_access_trace why),
TP_ARGS(cache_debug_id, ref, n_accesses, why),
TP_STRUCT__entry(
__field(unsigned int, cache )
__field(int, ref )
__field(int, n_accesses )
__field(enum fscache_access_trace, why )
),
TP_fast_assign(
__entry->cache = cache_debug_id;
__entry->ref = ref;
__entry->n_accesses = n_accesses;
__entry->why = why;
),
TP_printk("C=%08x %s r=%d a=%d",
__entry->cache,
__print_symbolic(__entry->why, fscache_access_traces),
__entry->ref,
__entry->n_accesses)
);
TRACE_EVENT(fscache_access_volume,
TP_PROTO(unsigned int volume_debug_id,
unsigned int cookie_debug_id,
int ref,
int n_accesses,
enum fscache_access_trace why),
TP_ARGS(volume_debug_id, cookie_debug_id, ref, n_accesses, why),
TP_STRUCT__entry(
__field(unsigned int, volume )
__field(unsigned int, cookie )
__field(int, ref )
__field(int, n_accesses )
__field(enum fscache_access_trace, why )
),
TP_fast_assign(
__entry->volume = volume_debug_id;
__entry->cookie = cookie_debug_id;
__entry->ref = ref;
__entry->n_accesses = n_accesses;
__entry->why = why;
),
TP_printk("V=%08x c=%08x %s r=%d a=%d",
__entry->volume,
__entry->cookie,
__print_symbolic(__entry->why, fscache_access_traces),
__entry->ref,
__entry->n_accesses)
);
TRACE_EVENT(fscache_access,
TP_PROTO(unsigned int cookie_debug_id,
int ref,
int n_accesses,
enum fscache_access_trace why),
TP_ARGS(cookie_debug_id, ref, n_accesses, why),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(int, ref )
__field(int, n_accesses )
__field(enum fscache_access_trace, why )
),
TP_fast_assign(
__entry->cookie = cookie_debug_id;
__entry->ref = ref;
__entry->n_accesses = n_accesses;
__entry->why = why;
),
TP_printk("c=%08x %s r=%d a=%d",
__entry->cookie,
__print_symbolic(__entry->why, fscache_access_traces),
__entry->ref,
__entry->n_accesses)
);
TRACE_EVENT(fscache_acquire,
@ -210,26 +406,21 @@ TRACE_EVENT(fscache_acquire,
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, parent )
__array(char, name, 8 )
__field(int, p_ref )
__field(int, p_n_children )
__field(u8, p_flags )
__field(unsigned int, volume )
__field(int, v_ref )
__field(int, v_n_cookies )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->parent = cookie->parent->debug_id;
__entry->p_ref = refcount_read(&cookie->parent->ref);
__entry->p_n_children = atomic_read(&cookie->parent->n_children);
__entry->p_flags = cookie->parent->flags;
memcpy(__entry->name, cookie->def->name, 8);
__entry->name[7] = 0;
__entry->volume = cookie->volume->debug_id;
__entry->v_ref = refcount_read(&cookie->volume->ref);
__entry->v_n_cookies = atomic_read(&cookie->volume->n_cookies);
),
TP_printk("c=%08x p=%08x pr=%d pc=%d pf=%02x n=%s",
__entry->cookie, __entry->parent, __entry->p_ref,
__entry->p_n_children, __entry->p_flags, __entry->name)
TP_printk("c=%08x V=%08x vr=%d vc=%d",
__entry->cookie,
__entry->volume, __entry->v_ref, __entry->v_n_cookies)
);
TRACE_EVENT(fscache_relinquish,
@ -239,9 +430,8 @@ TRACE_EVENT(fscache_relinquish,
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, parent )
__field(unsigned int, volume )
__field(int, ref )
__field(int, n_children )
__field(int, n_active )
__field(u8, flags )
__field(bool, retire )
@ -249,272 +439,58 @@ TRACE_EVENT(fscache_relinquish,
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->parent = cookie->parent->debug_id;
__entry->volume = cookie->volume->debug_id;
__entry->ref = refcount_read(&cookie->ref);
__entry->n_children = atomic_read(&cookie->n_children);
__entry->n_active = atomic_read(&cookie->n_active);
__entry->flags = cookie->flags;
__entry->retire = retire;
),
TP_printk("c=%08x r=%d p=%08x Nc=%d Na=%d f=%02x r=%u",
__entry->cookie, __entry->ref,
__entry->parent, __entry->n_children, __entry->n_active,
__entry->flags, __entry->retire)
TP_printk("c=%08x V=%08x r=%d U=%d f=%02x rt=%u",
__entry->cookie, __entry->volume, __entry->ref,
__entry->n_active, __entry->flags, __entry->retire)
);
TRACE_EVENT(fscache_enable,
TP_PROTO(struct fscache_cookie *cookie),
TRACE_EVENT(fscache_invalidate,
TP_PROTO(struct fscache_cookie *cookie, loff_t new_size),
TP_ARGS(cookie),
TP_ARGS(cookie, new_size),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(int, ref )
__field(int, n_children )
__field(int, n_active )
__field(u8, flags )
__field(loff_t, new_size )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->ref = refcount_read(&cookie->ref);
__entry->n_children = atomic_read(&cookie->n_children);
__entry->n_active = atomic_read(&cookie->n_active);
__entry->flags = cookie->flags;
__entry->new_size = new_size;
),
TP_printk("c=%08x r=%d Nc=%d Na=%d f=%02x",
__entry->cookie, __entry->ref,
__entry->n_children, __entry->n_active, __entry->flags)
TP_printk("c=%08x sz=%llx",
__entry->cookie, __entry->new_size)
);
TRACE_EVENT(fscache_disable,
TP_PROTO(struct fscache_cookie *cookie),
TRACE_EVENT(fscache_resize,
TP_PROTO(struct fscache_cookie *cookie, loff_t new_size),
TP_ARGS(cookie),
TP_ARGS(cookie, new_size),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(int, ref )
__field(int, n_children )
__field(int, n_active )
__field(u8, flags )
__field(loff_t, old_size )
__field(loff_t, new_size )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->ref = refcount_read(&cookie->ref);
__entry->n_children = atomic_read(&cookie->n_children);
__entry->n_active = atomic_read(&cookie->n_active);
__entry->flags = cookie->flags;
__entry->old_size = cookie->object_size;
__entry->new_size = new_size;
),
TP_printk("c=%08x r=%d Nc=%d Na=%d f=%02x",
__entry->cookie, __entry->ref,
__entry->n_children, __entry->n_active, __entry->flags)
);
TRACE_EVENT(fscache_osm,
TP_PROTO(struct fscache_object *object,
const struct fscache_state *state,
bool wait, bool oob, s8 event_num),
TP_ARGS(object, state, wait, oob, event_num),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, object )
__array(char, state, 8 )
__field(bool, wait )
__field(bool, oob )
__field(s8, event_num )
),
TP_fast_assign(
__entry->cookie = object->cookie->debug_id;
__entry->object = object->debug_id;
__entry->wait = wait;
__entry->oob = oob;
__entry->event_num = event_num;
memcpy(__entry->state, state->short_name, 8);
),
TP_printk("c=%08x o=%08d %s %s%sev=%d",
TP_printk("c=%08x os=%08llx sz=%08llx",
__entry->cookie,
__entry->object,
__entry->state,
__print_symbolic(__entry->wait,
{ true, "WAIT" },
{ false, "WORK" }),
__print_symbolic(__entry->oob,
{ true, " OOB " },
{ false, " " }),
__entry->event_num)
);
TRACE_EVENT(fscache_page,
TP_PROTO(struct fscache_cookie *cookie, struct page *page,
enum fscache_page_trace why),
TP_ARGS(cookie, page, why),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(pgoff_t, page )
__field(enum fscache_page_trace, why )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->page = page->index;
__entry->why = why;
),
TP_printk("c=%08x %s pg=%lx",
__entry->cookie,
__print_symbolic(__entry->why, fscache_page_traces),
__entry->page)
);
TRACE_EVENT(fscache_check_page,
TP_PROTO(struct fscache_cookie *cookie, struct page *page,
void *val, int n),
TP_ARGS(cookie, page, val, n),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(void *, page )
__field(void *, val )
__field(int, n )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->page = page;
__entry->val = val;
__entry->n = n;
),
TP_printk("c=%08x pg=%p val=%p n=%d",
__entry->cookie, __entry->page, __entry->val, __entry->n)
);
TRACE_EVENT(fscache_wake_cookie,
TP_PROTO(struct fscache_cookie *cookie),
TP_ARGS(cookie),
TP_STRUCT__entry(
__field(unsigned int, cookie )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
),
TP_printk("c=%08x", __entry->cookie)
);
TRACE_EVENT(fscache_op,
TP_PROTO(struct fscache_cookie *cookie, struct fscache_operation *op,
enum fscache_op_trace why),
TP_ARGS(cookie, op, why),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, op )
__field(enum fscache_op_trace, why )
),
TP_fast_assign(
__entry->cookie = cookie ? cookie->debug_id : 0;
__entry->op = op->debug_id;
__entry->why = why;
),
TP_printk("c=%08x op=%08x %s",
__entry->cookie, __entry->op,
__print_symbolic(__entry->why, fscache_op_traces))
);
TRACE_EVENT(fscache_page_op,
TP_PROTO(struct fscache_cookie *cookie, struct page *page,
struct fscache_operation *op, enum fscache_page_op_trace what),
TP_ARGS(cookie, page, op, what),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, op )
__field(pgoff_t, page )
__field(enum fscache_page_op_trace, what )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->page = page ? page->index : 0;
__entry->op = op->debug_id;
__entry->what = what;
),
TP_printk("c=%08x %s pg=%lx op=%08x",
__entry->cookie,
__print_symbolic(__entry->what, fscache_page_op_traces),
__entry->page, __entry->op)
);
TRACE_EVENT(fscache_wrote_page,
TP_PROTO(struct fscache_cookie *cookie, struct page *page,
struct fscache_operation *op, int ret),
TP_ARGS(cookie, page, op, ret),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, op )
__field(pgoff_t, page )
__field(int, ret )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->page = page->index;
__entry->op = op->debug_id;
__entry->ret = ret;
),
TP_printk("c=%08x pg=%lx op=%08x ret=%d",
__entry->cookie, __entry->page, __entry->op, __entry->ret)
);
TRACE_EVENT(fscache_gang_lookup,
TP_PROTO(struct fscache_cookie *cookie, struct fscache_operation *op,
void **results, int n, pgoff_t store_limit),
TP_ARGS(cookie, op, results, n, store_limit),
TP_STRUCT__entry(
__field(unsigned int, cookie )
__field(unsigned int, op )
__field(pgoff_t, results0 )
__field(int, n )
__field(pgoff_t, store_limit )
),
TP_fast_assign(
__entry->cookie = cookie->debug_id;
__entry->op = op->debug_id;
__entry->results0 = results[0] ? ((struct page *)results[0])->index : (pgoff_t)-1;
__entry->n = n;
__entry->store_limit = store_limit;
),
TP_printk("c=%08x op=%08x r0=%lx n=%d sl=%lx",
__entry->cookie, __entry->op, __entry->results0, __entry->n,
__entry->store_limit)
__entry->old_size,
__entry->new_size)
);
#endif /* _TRACE_FSCACHE_H */

5
include/trace/events/netfs.h
View File

@ -135,6 +135,7 @@ TRACE_EVENT(netfs_read,
__field(loff_t, start )
__field(size_t, len )
__field(enum netfs_read_trace, what )
__field(unsigned int, netfs_inode )
),
TP_fast_assign(
@ -143,12 +144,14 @@ TRACE_EVENT(netfs_read,
__entry->start = start;
__entry->len = len;
__entry->what = what;
__entry->netfs_inode = rreq->inode->i_ino;
),
TP_printk("R=%08x %s c=%08x s=%llx %zx",
TP_printk("R=%08x %s c=%08x ni=%x s=%llx %zx",
__entry->rreq,
__print_symbolic(__entry->what, netfs_read_traces),
__entry->cookie,
__entry->netfs_inode,
__entry->start, __entry->len)
);