Commit Graph

79 Commits

Author SHA1 Message Date
Paul Lawrence
5bb92dffc9 ANDROID: Incremental fs: Add FS_IOC_ENABLE_VERITY
Add FS_IOC_ENABLE_VERITY ioctl

When called, calculate measurement, validate signature against fsverity,
and set S_VERITY flag.

This does not (yet) preserve the verity status once the inode is
evicted.

Bug: 160634504
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I88af2721f650098accc72a64528c7d85b753c7f6
2021-03-01 16:15:32 +00:00
Paul Lawrence
695e0c5423 ANDROID: Incremental fs: Fix memory leak on closing file
Bug: 179271514
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Idc42d02b1df5ac84bdd04e728bfcca5f4cc5d07b
2021-03-01 16:15:18 +00:00
Paul Lawrence
400d6734bb ANDROID: Incremental fs: inotify on create mapped file
Bug: 175323815
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I670e8a7f4a68012d68718a431be3450646a614c0
2021-03-01 16:15:11 +00:00
Paul Lawrence
a699d0a2ab ANDROID: Incremental fs: inotify support
Bug: 175323815
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ife372fa2f10dd51f61def9feb461e965d276c6bf
2021-03-01 16:15:03 +00:00
Paul Lawrence
3d04fb2c95 ANDROID: Incremental fs: Make data validation failure a warn
Bug: 177234986
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I79b4273a050b8695b5810abd618fcb4437a05ce5
2021-01-14 16:03:13 +00:00
Paul Lawrence
4994a7321f ANDROID: Incremental fs: Free mount info after files are flushed
Bug: 177280103
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I24b0d4bf5353834900f868f65e7510529867b615
2021-01-14 16:03:02 +00:00
Paul Lawrence
a3c935a490 ANDROID: Incremental fs: Fix selinux issues
Bug: 177075428
Test: incfs_test passes
      atest GtsIncrementalInstallTestCases has only 8 failures
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I73accfc1982aec1cd7947996c25a23e4a97cfdac
2021-01-14 16:02:51 +00:00
Yurii Zubrytskyi
b79605a904 ANDROID: Incremental fs: fix .blocks_written
.blocks_writen file handling was missing some operations:
SELinux xattr handlers, safety checks for it being a
pseudo file etc.

This CL generalizes pseudo file handling so that all such
files work in a generic way and next time it should be
easier to add all operations at once.

Bug: 175823975
Test: incfs_tests pass
Change-Id: Id2b1936018c81c62c8ab4cdbaa8827e2679b513f
Signed-off-by: Yurii Zubrytskyi <zyy@google.com>
Signed-off-by: Paul Lawrence <paullawrence@google.com>
2021-01-14 16:02:38 +00:00
Paul Lawrence
dedf46b2a7 ANDROID: Incremental fs: Set credentials before reading/writing
Bug: 174692664
Test: incfs_test passes, incremental installs work with ag/13082306
Signed-off-by: Paul Lawrence <paullawrence@google.com>

Change-Id: Ib1c924bbaff759f58f7d83bad8e23d7224ba7ed9
2020-12-10 20:16:16 +00:00
Paul Lawrence
5ef8ab7172 ANDROID: Incremental fs: Change per UID timeouts to microseconds
Bug: 174495152
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Id76d2fec83a0eb7b70ad85f1fac81bf319563a66
2020-12-10 20:16:01 +00:00
Paul Lawrence
6fb25c4252 ANDROID: Incremental fs: Add v2 feature flag
Roll report_uid feature flag into v2 feature flag

Bug: 174478527
Test: Feature flag present on boot
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I41ee9715904560004e25cc83a5ccc1eb1bdd2b1f
2020-12-10 20:15:54 +00:00
Paul Lawrence
affa585cc0 ANDROID: Incremental fs: Add zstd feature flag
Bug: 174478527
Test: Boot, look for flag
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ib573b5420143bd177b50311a3e8cc3a7e8541b96
2020-12-10 20:15:46 +00:00
Paul Lawrence
95a43fc99f ANDROID: Incremental fs: Add zstd compression support
Bug: 160634783
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Iba28b535d2d5183859ffc721204b036434132d9b
2020-11-05 23:00:32 +00:00
Paul Lawrence
b1cc5f1b63 ANDROID: Incremental fs: Small improvements
Rmove bc_mutex used to protect metadata chain, now that is only
read at file open time
Remove certain unused mount options

Bug: 172482559
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Id70e5a5d08e5de79f391e19ea97e356f39a3ed51
2020-11-05 15:26:48 +00:00
Paul Lawrence
65a0f002c0 ANDROID: Incremental fs: Initialize mount options correctly
report_uid was not being initialized, leading to random behavior

Bug: 172480517
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ib121136d6f570f99e9060bdde9aa43ff2995514e
2020-11-05 15:26:31 +00:00
Paul Lawrence
23516028ff ANDROID: Incremental fs: Fix misuse of cpu_to_leXX and poll return
Found by sparse:

fs/incfs/format.c:416:21: warning: incorrect type in assignment (different base types)
fs/incfs/format.c:416:21:    expected restricted __le32 [assigned] [usertype] fh_flags
fs/incfs/format.c:416:21:    got int

fs/incfs/pseudo_files.c:925:25: warning: incorrect type in argument 4 (different base types)
fs/incfs/pseudo_files.c:925:25:    expected unsigned long long [usertype] size
fs/incfs/pseudo_files.c:925:25:    got restricted __le64 [addressable] [assigned] [usertype] size_attr_value
fs/incfs/pseudo_files.c:925:42: warning: incorrect type in argument 5 (different base types)
fs/incfs/pseudo_files.c:925:42:    expected unsigned long long [usertype] offset
fs/incfs/pseudo_files.c:925:42:    got restricted __le64 [usertype]
fs/incfs/pseudo_files.c:1111:24: warning: incorrect type in return expression (different base types)
fs/incfs/pseudo_files.c:1111:24:    expected restricted __poll_t
fs/incfs/pseudo_files.c:1111:24:    got int

Bug: 169258814
Fixes: Sparse errors introduced by 3f4938108a, 8334d69e65 and cb776f4576
Test: incfs_test passes, sparse shows no errors
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I48596e9521069fc77bf38c345a568529d61c77dc
2020-09-28 22:21:32 +00:00
Paul Lawrence
142953a07f ANDROID: Incremental fs: Add per UID read timeouts
Bug: 169056129
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I8cad9ee4095123bafba33abb65bbb339ba6ff8b5
2020-09-28 09:13:12 -07:00
Paul Lawrence
7e7bfa94e5 ANDROID: Incremental fs: Add .incomplete folder
Bug: 165929150
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ib6952391aea76bf0318cbad8da7a1276f8f9e8ba
2020-09-21 14:48:29 +00:00
Eric Biggers
87cca254c9 ANDROID: Incremental fs: Fix dangling else
Fix a compiler warning:

    fs/incfs/data_mgmt.c: In function ‘incfs_get_filled_blocks’:
    fs/incfs/data_mgmt.c:768:6: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]
      768 |   if (is_data_block_present(&dfb))
          |      ^

Fixes: 9cbdd375f6 ("ANDROID: Incremental fs: Fix filled block count from get filled blocks")
Change-Id: I5a331442155443c6f67e33f1bbaf5deae84a644b
Signed-off-by: Eric Biggers <ebiggers@google.com>
2020-09-17 19:50:57 -07:00
Eric Biggers
79581f9ee7 ANDROID: Incremental fs: Fix uninitialized variable
'error' isn't initialized before being returned.

Fixes: d4d1163e53 ("ANDROID: Incremental fs: Add INCFS_IOC_GET_BLOCK_COUNT")
Change-Id: I3082ac4a3f9d8632080a8e8c9221fda25ef152a0
Signed-off-by: Eric Biggers <ebiggers@google.com>
2020-09-17 19:47:36 -07:00
Paul Lawrence
9cbdd375f6 ANDROID: Incremental fs: Fix filled block count from get filled blocks
Bug: 165929150
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I8845adcafcc3a3f01730e8b5534fb25ea3d551db
2020-09-17 21:19:18 +00:00
Paul Lawrence
c0391ecb55 ANDROID: Incremental fs: Add hash block counts to IOC_IOCTL_GET_BLOCK_COUNT
Bug: 166638631
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I0061a855ec563de8df40ba0b35292e99be33c6c0
2020-09-16 15:44:06 -07:00
Paul Lawrence
d4d1163e53 ANDROID: Incremental fs: Add INCFS_IOC_GET_BLOCK_COUNT
Bug: 166638631
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ia7a8cab87688fc401f0719df84fe79ea75887692
2020-09-16 18:05:34 +00:00
Paul Lawrence
c178b96af9 ANDROID: Incremental fs: Make compatible with existing files
Bug: 166638631
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I3b4f61bf881dccd50f05f42326370421945b68db
2020-09-16 15:51:59 +00:00
Paul Lawrence
ae73f6551a ANDROID: Incremental fs: Remove block HASH flag
Bug: 166638631
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I98e1efb9913d7c920ecaa76d29c8cf36219ff930
2020-09-10 10:13:17 -07:00
Paul Lawrence
4da1cc2822 ANDROID: Incremental fs: Remove back links and crcs
Bug: 166638631
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ia6061f8985db85a66fce0f2f09e1305a21753dd6
2020-09-10 10:13:17 -07:00
Paul Lawrence
900900ef89 ANDROID: Incremental fs: Remove attributes from file
Bug: 166638631
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I0f753929f6b5b884f829cd3e6e8d4e6f94ab3a46
2020-09-10 10:13:17 -07:00
Paul Lawrence
cb776f4576 ANDROID: Incremental fs: Add .blocks_written file
Bug: 162856396
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I942582218cdc7741bcff2f264960b76cdfa1bb36
2020-09-10 10:13:08 -07:00
Paul Lawrence
8334d69e65 ANDROID: Incremental fs: Separate pseudo-file code
Also fixed two bugs in the process:

is_pseudo_filename was not previously checking for .log, so an attempt
to create a .log would succeed.

All ioctls could be called on all files. ioctls now set on the correct
files.

Bug: 162856396
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I3f1e87d018836f51a97897880dd70181db4f7169
2020-08-26 18:59:29 +00:00
Paul Lawrence
7ab6cf0fec ANDROID: Incremental fs: Add UID to pending_read
Test: incfs_test passes
Bug: 160634477
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Iaf817cf1f7ccd0109b2114b425ea7f26718345ab
2020-08-19 17:13:16 +00:00
Paul Lawrence
3f4938108a ANDROID: Incremental fs: Create mapped file
Bug: 160634482
Test: incfs_test passes
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ic2ac8dfccd60f6c9c72c38bf323997fce7546c1c
2020-08-18 12:37:59 -07:00
mtk81325
7c02f94255 ANDROID: Incremental fs: fix magic compatibility again
It's still magic number issue which cannot be compatible with
arm-32 platform, although we try to fix it in Iae4f3877444
("ANDROID: Incremental fs: magic number compatible 32-bit"),
there is still incompatible scenario, such as: get_incfs_node(),
it will return NULL then kernel exception will be trigger because
of NULL pointer access. (inode_set() -> get_incfs_node(), then used
node->xxx directly)

We change magic number directly, otherwise, we must fix above issues one by one.

Bug: 159772865
Fixes: Iae4f3877444("ANDROID: Incremental fs: magic number compatible 32-bit")
Signed-off-by: Peng Zhou <Peng.Zhou@mediatek.com>
Signed-off-by: mtk81325 <peng.zhou@mediatek.com>
Change-Id: I71f279c1bb55ea296ab33a47644f30df4a9f60a6
2020-08-18 17:12:58 +00:00
Akilesh Kailash
22b27c439f ANDROID: Incremental fs: Don't allow renaming .index directory.
Add additional check to verify dentry is hashed after vfs_mkdir()
is successful.

Bug: 148423333
Test: kernel seftest - incfs_test, manual test removing .index when
mounted

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I7d2d71d480db4f02b7f3a347b5734c6643843594
2020-08-13 02:46:00 +00:00
Paul Lawrence
b59b963a7d ANDROID: Incremental fs: Fix incfs to work on virtio-9p
Test: incfs_test on a virtio 9p drive. Note test 6 fails still,
but I think this is a result of caching directory entries over a
network file system.
Bug: 161802292
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I6986fb3e9b403181cf81024046f394960caf4620
2020-08-04 17:06:31 +00:00
Akilesh Kailash
f39f1b550a ANDROID: Incremental fs: Use R/W locks to read/write segment blockmap.
Use Read-Write locks for reading/writing segment in blockmap.
This should allow parallel reads when there are
multiple reads within same segment.

A small optimization in pending_reads_read(). Since
incfs_collect_pending_reads() already iterate to
populate buffer, new_max_sn - highest serial number
among all the pending read buffer can be done in the same
loop instead of looping again in pending_reads_read().

Bug: 161566104
Test: kernel selftest - incfs_test and incfs_perf

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id00376b0e4cb8c0c0bc8264cdddd6f38c4aa85f0
2020-07-28 05:11:50 +00:00
Paul Lawrence
a95005100d ANDROID: Incremental fs: Remove unnecessary dependencies
Bug: 162260581
Test: builds with defconfig
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I743fa7a989f5a464f755a22b6a03299fb462e74c
2020-07-27 21:23:55 +00:00
Paul Lawrence
7ec98e58bd ANDROID: Incremental fs: Remove annoying pr_debugs
Bug: 162255528
Test: incfs_test passes with fewer annoying debug messages
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ib347d3d5dd8a3c5545a0760fc674147211aa8c9d
2020-07-27 14:05:59 -07:00
Akilesh Kailash
b0fa24d335 ANDROID: Incremental fs: dentry_revalidate should not return -EBADF.
Fix of bug 161565969 added EBADF when backing path doesn't
exist. However, .log and .pending reads doesn't have
backing path causing incfs_test to fail.

Bug: 162243475
Test: incfs_test on QEMU passes

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Iba303bbc42b431fe4a4fbacb40584faaff4b14b7
2020-07-27 18:47:26 +00:00
Akilesh Kailash
000d95ac2d ANDROID: Incremental fs: Fix minor bugs
1: Invoke kunmap(page) in error path
2: Validate NULL checks at few places in the code.
3: path_put() should not be invoked if path entry is null.
   Although path_put() checks for NULL condition internally,
   caller should gracefully handle it.

Bug: 161565969
Test: kernel selftest - incfs_test, incfs_perf

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ie4dfaaba4b09f4798d492f8a25dd9dcc8da89e51
2020-07-24 19:13:44 +00:00
Akilesh Kailash
0f406eaea5 ANDROID: Incremental fs: RCU locks instead of mutex for pending_reads.
Use RCU locks instead of pending_reads_mutex.
Current mutex is taking lock on entire mount_info
structure which seems a heavy operation.

Following fields of mount_info structure
are protected through spinlocks for multiple
writers and are RCU safe for readers:

- reads_list_head
- mi_pending_reads_count
- mi_last_pending_read_number
- data_file_segment.reads_list_head

We could probably use atomic_inc/atomic_dec for
mi_pending_reads_count and mi_last_pending_read_number
which can futher cut down spin_locks at couple of more places,
thereby only the list addition and removal can protected
by spinlock. This CL doesn't address it.

Bug: 161565969
Test: kernel selftest incfs_test and incfs_perf

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Iad7439657016764dce25d64c8b3df69b930452bc
2020-07-24 17:35:04 +00:00
mtk81325
839511af20 ANDROID: Incremental fs: magic number compatible 32-bit
Incfs's magic is bigger than 32-bit, but super block structure's
s_magic is unsigned long which is 32-bit in ARM 32-bit platform.

Do the cast for magic!

Bug: 159772865
Signed-off-by: Peng Zhou <Peng.Zhou@mediatek.com>
Signed-off-by: mtk81325 <peng.zhou@mediatek.com>
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Iae4f38774440c7d6ae44529d4f0f8ebb2ec5dacc
2020-07-24 16:54:44 +00:00
Greg Kroah-Hartman
3be0477f5a ANDROID: Incremental fs: fix up attempt to copy structures with READ/WRITE_ONCE
READ/WRITE_ONCE are for atomic data types, not for structures.  Fix this
up by doing a memcpy to make it explicit just how messy this copy is...

This fixes a build error on 5.8-rc1, as things are more strict, odds are
it's also wrong in other kernel versions as well...

Cc: Daniel Mentz <danielmentz@google.com>
Cc: Paul Lawrence <paullawrence@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I7ecd3d05bd94c936dd5e69c63028458786f37a78
2020-06-25 15:20:35 +00:00
Daniel Mentz
83c1d9116e ANDROID: Incremental fs: Remove dependency on PKCS7_MESSAGE_PARSER
Incremental fs appears to not depend on pkcs7 anymore.

Bug: 151584760
Signed-off-by: Daniel Mentz <danielmentz@google.com>
Change-Id: I809b4b5651d84ca70fd8bf837765e33df8547418
Signed-off-by: Paul Lawrence <paullawrence@google.com>
2020-06-08 17:03:32 +00:00
Paul Lawrence
60bc6eaf98 ANDROID: Incremental fs: Fix four error-path bugs
Test: incfs_test passes
Bug: 158242405
Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ib53e867fb2681489f720f6255354c1bce1d33997
2020-06-08 14:24:39 +00:00
Paul Lawrence
21e6d932da ANDROID: Incremental fs: Cache successful hash calculations
Bug: 155996534
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Ic508e6fa07c90decb29e07647dd3b0fc4d243ce8
2020-06-04 21:17:47 +00:00
Paul Lawrence
eacb5b3579 Revert "ANDROID: Incremental fs: Avoid continually recalculating hashes"
This reverts commit ab185e45f6.

This change used the PageChecked flag to mark the Merkle tree as
checked. However, f2fs uses this internally. This caused file system
hangs on devices after installs.

Test: incfs_test passes, installs no longer hang
Bug: 157589629
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I980a700d65eb4f4a77434715d61dda4b8e80658c
2020-05-27 21:34:03 +00:00
Yurii Zubrytskyi
a34bcc8b20 ANDROID: Incremental fs: wake up log pollers less often
Waking up the waiters accounts for 80+% of the total logging
time, and about 40% of overall read_single_page() with no
signature verification. By throttling it to once every 16ms
we get back all read performance, reduce the waiter's CPU
usage and still leave it enough time to pull the logs out.

Bug: 155996534
Test: adb install megacity.apk & dd from the installed apk

Signed-off-by: Yurii Zubrytskyi <zyy@google.com>
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I4a118dc226d7ca318cf099ba3e239f0120bb23c2
2020-05-14 14:32:55 +00:00
Paul Lawrence
f6691cbdb6 ANDROID: Incremental fs: Fix scheduling while atomic error
If an incfs file is created, then the file system is sync'd,on opening
the incfs file inode_set reads the size from the backing file from
within iget5_locked, causing this error.

Test: incfs_test passes, this no longer occurs
Bug: 156413528
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I8939c4afa514d39d251c044d7680cfc69272669e
2020-05-13 13:28:59 +00:00
Paul Lawrence
ab185e45f6 ANDROID: Incremental fs: Avoid continually recalculating hashes
With a verified file (use incfs_perf to create a verified file), throughput
measured using dd after dropping caches increases from 200M/s to 290M/s

Test: incfs_test passes
Bug: 155996534
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: I7abb5ad92e4167f82f3452acc9db322fec8307dd
2020-05-12 20:09:47 +00:00
Paul Lawrence
3e4fa206ce ANDROID: Incremental fs: Fix issues with very large files
Test: incfs_test passes
Bug: 155590527
Signed-off-by: Paul Lawrence <paullawrence@google.com>
Change-Id: Iaecfcd40e8c089d11b34c7aff2090fbfe0c36219
2020-05-04 23:37:33 +00:00