Line data Source code
1 : #ifndef HEADER_fd_src_flamenco_runtime_program_fd_program_cache_h
2 : #define HEADER_fd_src_flamenco_runtime_program_fd_program_cache_h
3 :
4 : #include "../../fd_flamenco_base.h"
5 : #include "../fd_acc_mgr.h"
6 : #include "../context/fd_exec_slot_ctx.h"
7 : #include "../../vm/syscall/fd_vm_syscall.h"
8 : #include "../fd_system_ids.h"
9 :
10 : /* fd_program_cache contains the core logic for the program cache's
11 : behavior, including accesses, insertions, updates, and verifies.
12 : Specifically, the program cache operates in a lazy manner where
13 : programs are only inserted / reverified when they are referenced
14 : in a transaction.
15 :
16 : The program cache lives in Funk and is fork-aware. The size of the
17 : program cache is bounded by the size of the Funk instance.
18 :
19 : On bootup, the program cache starts out empty. As programs (defined
20 : as accounts owned by one of the BPF loaders that can be invoked
21 : by the user) are referenced as account keys within transactions
22 : (either statically or through address lookup tables),
23 : `fd_program_cache_update_program()` accepts a pubkey as input and
24 : will attempt to either insert the program into the cache (performing
25 : sBPF + ELF validations) if missing, or reverify the program if
26 : needed.
27 :
28 : During transaction execution, we accumulate any program pubkeys that
29 : are deployed, extended, or upgraded (basically, any calls to
30 : `fd_deploy_program()` in the BPF loaders) since those instructions
31 : may have changed the program's executable data (closed / retracted
32 : programs also do, but see comments below on how we handle that).
33 : After a transaction is executed successfully, we iterate through
34 : these accumulated program keys and queue them for reverification.
35 : To mark a program for reverification, we simply update the
36 : `last_slot_modified` field to the current slot. When the program is
37 : next referenced in a transaction, `fd_program_cache_update_program()`
38 : will check if the program was modified since the last time it was
39 : reverified, and reverify the program accordingly to update the cache
40 : entry's sBPF info, calldests, etc. Once the program has been
41 : reverified, the entry's `last_slot_verified` field will be updated
42 : to the current slot.
43 :
44 : When a program is reverified and updated in the cache, we clone the
45 : record down to the current funk transaction from an ancestor, and
46 : then modify the record within the current funk transaction.
47 :
48 : A program cache entry may also be reverified after crossing an
49 : epoch boundary, even if it was not modified recently. This is because
50 : the network's active feature set may have changed, and any previously
51 : {valid,invalid} programs may now be {valid,invalid}. Therefore,
52 : if a program is referenced in a transaction that has not been
53 : reverified since the last epoch (regardless of it having been
54 : modified or not), it will be reverified and updated.
55 :
56 : If a program fails verification due to invalid ELF headers / sBPF
57 : loading failures, then we update the `failed_verification` tag,
58 : set `last_slot_verified` to the current slot, and set
59 : `last_slot_modified` to 0. A future program upgrade / network feature
60 : set change could make this program valid again.
61 :
62 : A key invariant with the program cache design is that it does not
63 : evict any entries - it only grows through the lifetime of the
64 : client's execution. Because of this invariant, we do not touch cache
65 : entries for programs which are closed, and instead let
66 : transaction-level account loading and BPF loader program checks
67 : catch cases where a user tries to invoke a closed program.
68 :
69 : Another key invariant is that for any given program, we will insert /
70 : update its cache entry at most ONCE per slot, and this will happen
71 : the FIRST time the program is referenced in a transaction before it
72 : is dispatched to the exec tiles. This is important because it ensures
73 : that the replay tile does not race with the exec tiles by updating
74 : the cache entry for a program while an exec tile is already executing
75 : it. Even if a program is upgraded in the same slot after it has been
76 : reverified, the network forbids invoking programs in the same slot as
77 : they are deployed / upgraded, so we can wait for a future slot to
78 : update the program cache entry when it is invoked next.
79 :
80 : EDGE CASES (and how we handle them):
81 : - Deploying programs
82 : - Deploying a program
83 : - `fd_program_cache_queue_program_for_reverification()` will
84 : not do anything because the program does not exist in the
85 : cache yet, and so there is nothing to update. The next time
86 : the program is referenced in a transaction,
87 : `fd_program_cache_update_program()` will see that the program
88 : is missing in the cache and will perform ELF / sBPF
89 : verifications and insert the entry into the cache.
90 : - Deploying and invoking in the same slot
91 : - BPF loader checks will fail because the program account's
92 : slot value will be equal to the current slot, which
93 : gets caught by the `DelayedVisibility` checks.
94 : - Upgrading programs
95 : - Upgrading a program
96 : - The program account will have been referenced as writable
97 : in the transaction, and thus
98 : `fd_program_cache_queue_program_for_reverification()` will
99 : update the `last_slot_modified` to queue the program
100 : for reverification the next time it is referenced in a
101 : transaction.
102 : - Upgrading and invoking in the same transaction
103 : - Same as "deploy" case
104 : - Closing programs
105 : - Closing a program
106 : - We do not touch the program cache here. We let account
107 : loading / BPF loader checks handle cases where a user may try
108 : to invoke a closed program.
109 : - Closing + invoking a program in the same transaction
110 : - The program account's state will be set to uninitialized /
111 : retracted, so any future instructions that invoke the program
112 : will fail when the BPF loader checks the account state.
113 : - Closing + invoking a program in separate transactions
114 : - The program account's owner will be set to the system program
115 : and thus fail account loading checks.
116 :
117 : TL;DR: Deploys and upgrades are treated the same way - if the cache
118 : entry exists, it's queued for reverification. If it doesn't, the
119 : program cache will verify and add it to the cache the next time it's
120 : invoked. Cases where the program is closed / retracted are not
121 : explicitly handled.
122 :
123 : Q: Is the program cache concurrency-safe?
124 : A: Yes. In any given slot, the replay tile first iterates through
125 : each transaction in a single-threaded manner. For each
126 : transaction, it will read the accounts and will call
127 : `fd_program_cache_update_program()` for each program, and then
128 : dispatch the transaction to the exec tiles. This means
129 : that the first time any program is referenced in any transaction
130 : in the slot, the replay tile will update the associated program
131 : cache entry before dispatching the transaction to any of the exec
132 : tiles. Furthermore, we are not allowed to insert / reverify a
133 : program cache entry more than once within a single slot. This
134 : guarantees that any read / write accesses for a particular program
135 : in the program cache by the exec / writer tiles will occur after
136 : the cache entries have been processed by the replay tile in any
137 : given slot. Furthermore, if the program was upgraded, the writer
138 : tile simply updates a single header in the existing program cache
139 : entry `last_slot_modified`, which is behind a blocking write lock.
140 : Note that if there is read-write or write-write-contention
141 : between two transactions for any accounts, the scheduler will
142 : ensure that those two transactions are scheduled and finalized
143 : in a serial manner. */
144 :
145 : /* `fd_program_cache_entry` defines the structure for a single
146 : program cache entry. */
147 : struct fd_program_cache_entry {
148 : ulong magic;
149 :
150 : /* For any programs that fail verification, we retain this flag for
151 : to prevent any reverification attempts for the remainder of the
152 : epoch / until the program is modified again (instead of removing
153 : them from the cache). When `failed_verification` is set,
154 : the values of all other fields except for
155 : `last_slot_verified` are undefined. Any invocations of a
156 : program that fails verification will continue to fail until the
157 : program is queued for reverification by an eligible BPF loader
158 : instruction (e.g. the program is upgraded). */
159 : uchar failed_verification;
160 :
161 : /* Stores the last slot the program was modified. This field is
162 : updated at the end of a transaction for a program account that was
163 : referenced as a writable account within an instruction. For any
164 : programs that are freshly added to the cache, this field is set to
165 : 0. */
166 : ulong last_slot_modified;
167 :
168 : /* Stores the last slot verification checks were ran for a program.
169 : Programs are reverified if they are mentioned in the current
170 : transaction, and if one of the following are true:
171 : - It is the first time they are referenced in a transaction in the
172 : current epoch
173 : - The program was a writable account in a transaction
174 :
175 : We reverify referenced programs at least once every epoch
176 : regardless of whether they were modified or not because changes
177 : in the active feature set may cause existing deployed programs
178 : to be invalided (e.g. stricter ELF / VM / sBPF checks). */
179 : ulong last_slot_verified;
180 :
181 : ulong entry_pc;
182 : ulong text_cnt;
183 : ulong text_off;
184 : ulong text_sz;
185 :
186 : ulong rodata_sz;
187 :
188 : /* We keep the pointer to the calldests raw memory around, so that we can easily copy the entire
189 : data structures (including the private header) later. */
190 : void * calldests_shmem;
191 : fd_sbpf_calldests_t * calldests;
192 : uchar * rodata;
193 :
194 : /* SBPF version, SIMD-0161 */
195 : ulong sbpf_version;
196 : };
197 : typedef struct fd_program_cache_entry fd_program_cache_entry_t;
198 :
199 : /* arbitrary unique value, in this case
200 : echo -n "fd_program_cache_entry" | sha512sum | head -c 16 */
201 33 : #define FD_PROGRAM_CACHE_ENTRY_MAGIC 0xb45640baf006ddf6
202 :
203 : FD_PROTOTYPES_BEGIN
204 :
205 : fd_program_cache_entry_t *
206 : fd_program_cache_entry_new( void * mem,
207 : fd_sbpf_elf_info_t const * elf_info,
208 : ulong last_slot_modified,
209 : ulong last_slot_verified );
210 :
211 : ulong
212 : fd_program_cache_entry_footprint( fd_sbpf_elf_info_t const * elf_info );
213 :
214 : /* Loads a single program cache entry for a given pubkey. Returns 0 on
215 : success and -1 on failure. On success, `*cache_entry` holds a pointer
216 : to the program cache entry. */
217 : int
218 : fd_program_cache_load_entry( fd_funk_t const * funk,
219 : fd_funk_txn_t const * funk_txn,
220 : fd_pubkey_t const * program_pubkey,
221 : fd_program_cache_entry_t const ** cache_entry );
222 :
223 : /* Parses the programdata from a program account. Returns a pointer to
224 : the program data and sets `out_program_data_len` on success. Returns
225 : NULL on failure or if the program account is not owned by a BPF
226 : loader program ID, and leaves `out_program_data_len` in an undefined
227 : state. Reasons for failure vary on the loader version. See the
228 : respective functions in this file for more details.
229 :
230 : This is essentially load_program_with_pubkey(), sans the ELF parsing
231 : and whatnot which is done in load_program_from_bytes(). */
232 : uchar const *
233 : fd_program_cache_get_account_programdata( fd_funk_t const * funk,
234 : fd_funk_txn_t const * funk_txn,
235 : fd_txn_account_t const * program_acc,
236 : ulong * out_program_data_len,
237 : fd_spad_t * runtime_spad );
238 :
239 : /* Updates the program cache for a single program. This function is
240 : called for every program that is referenced in a transaction, plus
241 : every single account in a lookup table referenced in the transaction.
242 : This function...
243 : - Accepts a pubkey and reads the programdata from the account
244 : - Creates a program cache entry for the program if it doesn't exist
245 : in the cache already
246 : - Reverifies programs if either...
247 : - The program was recently modified in a STRICTLY PRIOR SLOT
248 : - The program has not been verified yet for the current epoch
249 : - Invalidated programs that fail ELF / sBPF verification
250 : - Updates the program cache entry for the program after
251 : reverification (syscalls, calldests, etc)
252 :
253 : With this design, the program cache is designed to only grow as new
254 : programs are deployed / invoked. If a program fails verification, it
255 : stays in the cache so that repeated calls won't DOS the validator by
256 : forcing reverifications (since we won't be able to distinguish failed
257 : verifications from new deployments).
258 :
259 : When a program is reverified (and the cache entry already exists in
260 : some ancestor funk transaction), we clone the record down to the
261 : current funk transaction and acquire a blocking write lock on the
262 : cloned funk record. */
263 : void
264 : fd_program_cache_update_program( fd_exec_slot_ctx_t * slot_ctx,
265 : fd_pubkey_t const * program_key,
266 : fd_spad_t * runtime_spad );
267 :
268 : /* Queues a single program account for reverification. This function
269 : queries the cache for an existing entry and queues it for
270 : reverification by setting the `last_slot_modified` field in the
271 : program cache entry to the current slot. If the cache entry
272 : for the program does not exist yet (e.g. newly deployed programs),
273 : this function does nothing and instead,
274 : `fd_program_cache_publish_failed_verification_rec()` will insert
275 : the program into the cache.
276 :
277 : If the record exists in the program cache, this function will
278 : acquire a write-lock on the program cache entry. */
279 : void
280 : fd_program_cache_queue_program_for_reverification( fd_funk_t * funk,
281 : fd_funk_txn_t * funk_txn,
282 : fd_pubkey_t const * program_key,
283 : ulong current_slot );
284 :
285 : FD_PROTOTYPES_END
286 :
287 : #endif /* HEADER_fd_src_flamenco_runtime_program_fd_program_cache_h */
|
