Line data Source code
1 : #define _GNU_SOURCE
2 :
3 : /* Let's say there was a computer, the "leader" computer, that acted as
4 : a bank. Users could send it messages saying they wanted to deposit
5 : money, or transfer it to someone else.
6 :
7 : That's how, for example, Bank of America works but there are problems
8 : with it. One simple problem is: the bank can set your balance to
9 : zero if they don't like you.
10 :
11 : You could try to fix this by having the bank periodically publish the
12 : list of all account balances and transactions. If the customers add
13 : unforgeable signatures to their deposit slips and transfers, then
14 : the bank cannot zero a balance without it being obvious to everyone.
15 :
16 : There's still problems. The bank can't lie about your balance now or
17 : take your money, but it can just not accept deposits on your behalf
18 : by ignoring you.
19 :
20 : You could fix this by getting a few independent banks together, lets
21 : say Bank of America, Bank of England, and Westpac, and having them
22 : rotate who operates the leader computer periodically. If one bank
23 : ignores your deposits, you can just wait and send them to the next
24 : one.
25 :
26 : This is Solana.
27 :
28 : There's still problems of course but they are largely technical. How
29 : do the banks agree who is leader? How do you recover if a leader
30 : misbehaves? How do customers verify the transactions aren't forged?
31 : How do banks receive and publish and verify each others work quickly?
32 : These are the main technical innovations that enable Solana to work
33 : well.
34 :
35 : What about Proof of History?
36 :
37 : One particular niche problem is about the leader schedule. When the
38 : leader computer is moving from one bank to another, the new bank must
39 : wait for the old bank to say it's done and provide a final list of
40 : balances that it can start working off of. But: what if the computer
41 : at the old bank crashes and never says its done?
42 :
43 : Does the new leader just take over at some point? What if the new
44 : leader is malicious, and says the past thousand leaders crashed, and
45 : there have been no transactions for days? How do you check?
46 :
47 : This is what Proof of History solves. Each bank in the network must
48 : constantly do a lot of busywork (compute hashes), even when it is not
49 : leader.
50 :
51 : If the prior thousand leaders crashed, and no transactions happened
52 : in an hour, the new leader would have to show they did about an hour
53 : of busywork for everyone else to believe them.
54 :
55 : A better name for this is proof of skipping. If a leader is skipping
56 : slots (building off of a slot that is not the direct parent), it must
57 : prove that it waited a good amount of time to do so.
58 :
59 : It's not a perfect solution. For one thing, some banks have really
60 : fast computers and can compute a lot of busywork in a short amount of
61 : time, allowing them to skip prior slot(s) anyway. But: there is a
62 : social component that prevents validators from skipping the prior
63 : leader slot. It is easy to detect when this happens and the network
64 : could respond by ignoring their votes or stake.
65 :
66 : You could come up with other schemes: for example, the network could
67 : just use wall clock time. If a new leader publishes a block without
68 : waiting 400 milliseconds for the prior slot to complete, then there
69 : is no "proof of skipping" and the nodes ignore the slot.
70 :
71 : These schemes have a problem in that they are not deterministic
72 : across the network (different computers have different clocks), and
73 : so they will cause frequent forks which are very expensive to
74 : resolve. Even though the proof of history scheme is not perfect,
75 : it is better than any alternative which is not deterministic.
76 :
77 : With all that background, we can now describe at a high level what
78 : this PoH tile actually does,
79 :
80 : (1) Whenever any other leader in the network finishes a slot, and
81 : the slot is determined to be the best one to build off of, this
82 : tile gets "reset" onto that block, the so called "reset slot".
83 :
84 : (2) The tile is constantly doing busy work, hash(hash(hash(...))) on
85 : top of the last reset slot, even when it is not leader.
86 :
87 : (3) When the tile becomes leader, it continues hashing from where it
88 : was. Typically, the prior leader finishes their slot, so the
89 : reset slot will be the parent one, and this tile only publishes
90 : hashes for its own slot. But if prior slots were skipped, then
91 : there might be a whole chain already waiting.
92 :
93 : That's pretty much it. When we are leader, in addition to doing
94 : busywork, we publish ticks and microblocks to the shred tile. A
95 : microblock is a non-empty group of transactions whose hashes are
96 : mixed-in to the chain, while a tick is a periodic stamp of the
97 : current hash, with no transactions (nothing mixed in). We need
98 : to send both to the shred tile, as ticks are important for other
99 : validators to verify in parallel.
100 :
101 : As well, the tile should never become leader for a slot that it has
102 : published anything for, otherwise it may create a duplicate block.
103 :
104 : Some particularly common misunderstandings:
105 :
106 : - PoH is critical to security.
107 :
108 : This largely isn't true. The target hash rate of the network is
109 : so slow (1 hash per 500 nanoseconds) that a malicious leader can
110 : easily catch up if they start from an old hash, and the only
111 : practical attack prevented is the proof of skipping. Most of the
112 : long range attacks in the Solana whitepaper are not relevant.
113 :
114 : - PoH keeps passage of time.
115 :
116 : This is also not true. The way the network keeps time so it can
117 : decide who is leader is that, each leader uses their operating
118 : system clock to time 400 milliseconds and publishes their block
119 : when this timer expires.
120 :
121 : If a leader just hashed as fast as they could, they could publish
122 : a block in tens of milliseconds, and the rest of the network
123 : would happily accept it. This is why the Solana "clock" as
124 : determined by PoH is not accurate and drifts over time.
125 :
126 : - PoH prevents transaction reordering by the leader.
127 :
128 : The leader can, in theory, wait until the very end of their
129 : leader slot to publish anything at all to the network. They can,
130 : in particular, hold all received transactions for 400
131 : milliseconds and then reorder and publish some right at the end
132 : to advantage certain transactions.
133 :
134 : You might be wondering... if all the PoH chain is helping us do is
135 : prove that slots were skipped correctly, why do we need to "mix in"
136 : transactions to the hash value? Or do anything at all for slots
137 : where we don't skip the prior slot?
138 :
139 : It's a good question, and the answer is that this behavior is not
140 : necessary. An ideal implementation of PoH have no concept of ticks
141 : or mixins, and would not be part of the TPU pipeline at all.
142 : Instead, there would be a simple field "skip_proof" on the last
143 : shred we send for a slot, the hash(hash(...)) value. This field
144 : would only be filled in (and only verified by replayers) in cases
145 : where the slot actually skipped a parent.
146 :
147 : Then what is the "clock? In Solana, time is constructed as follows:
148 :
149 : HASHES
150 :
151 : The base unit of time is a hash. Hereafter, any values whose
152 : units are in hashes are called a "hashcnt" to distinguish them
153 : from actual hashed values.
154 :
155 : Agave generally defines a constant duration for each tick
156 : (see below) and then varies the number of hashcnt per tick, but
157 : as we consider the hashcnt the base unit of time, Firedancer and
158 : this PoH implementation defines everything in terms of hashcnt
159 : duration instead.
160 :
161 : In mainnet-beta, testnet, and devnet the hashcnt ticks over
162 : (increments) every 100 nanoseconds. The hashcnt rate is
163 : specified as 500 nanoseconds according to the genesis, but there
164 : are several features which increase the number of hashes per
165 : tick while keeping tick duration constant, which make the time
166 : per hashcnt lower. These features up to and including the
167 : `update_hashes_per_tick6` feature are activated on mainnet-beta,
168 : devnet, and testnet, and are described in the TICKS section
169 : below.
170 :
171 : Other chains and development environments might have a different
172 : hashcnt rate in the genesis, or they might not have activated
173 : the features which increase the rate yet, which we also support.
174 :
175 : In practice, although each validator follows a hashcnt rate of
176 : 100 nanoseconds, the overall observed hashcnt rate of the
177 : network is a little slower than once every 100 nanoseconds,
178 : mostly because there are gaps and clock synchronization issues
179 : during handoff between leaders. This is referred to as clock
180 : drift.
181 :
182 : TICKS
183 :
184 : The leader needs to periodically checkpoint the hash value
185 : associated with a given hashcnt so that they can publish it to
186 : other nodes for verification.
187 :
188 : On mainnet-beta, testnet, and devnet this occurs once every
189 : 62,500 hashcnts, or approximately once every 6.4 microseconds.
190 : This value is determined at genesis time, and according to the
191 : features below, and could be different in development
192 : environments or on other chains which we support.
193 :
194 : Due to protocol limitations, when mixing in transactions to the
195 : proof-of-history chain, it cannot occur on a tick boundary (but
196 : can occur at any other hashcnt).
197 :
198 : Ticks exist mainly so that verification can happen in parallel.
199 : A verifier computer, rather than needing to do hash(hash(...))
200 : all in sequence to verify a proof-of-history chain, can do,
201 :
202 : Core 0: hash(hash(...))
203 : Core 1: hash(hash(...))
204 : Core 2: hash(hash(...))
205 : Core 3: hash(hash(...))
206 : ...
207 :
208 : Between each pair of tick boundaries.
209 :
210 : Solana sometimes calls the current tick the "tick height",
211 : although it makes more sense to think of it as a counter from
212 : zero, it's just the number of ticks since the genesis hash.
213 :
214 : There is a set of features which increase the number of hashcnts
215 : per tick. These are all deployed on mainnet-beta, devnet, and
216 : testnet.
217 :
218 : name: update_hashes_per_tick
219 : id: 3uFHb9oKdGfgZGJK9EHaAXN4USvnQtAFC13Fh5gGFS5B
220 : hashes per tick: 12,500
221 : hashcnt duration: 500 nanos
222 :
223 : name: update_hashes_per_tick2
224 : id: EWme9uFqfy1ikK1jhJs8fM5hxWnK336QJpbscNtizkTU
225 : hashes per tick: 17,500
226 : hashcnt duration: 357.142857143 nanos
227 :
228 : name: update_hashes_per_tick3
229 : id: 8C8MCtsab5SsfammbzvYz65HHauuUYdbY2DZ4sznH6h5
230 : hashes per tick: 27,500
231 : hashcnt duration: 227.272727273 nanos
232 :
233 : name: update_hashes_per_tick4
234 : id: 8We4E7DPwF2WfAN8tRTtWQNhi98B99Qpuj7JoZ3Aikgg
235 : hashes per tick: 47,500
236 : hashcnt duration: 131.578947368 nanos
237 :
238 : name: update_hashes_per_tick5
239 : id: BsKLKAn1WM4HVhPRDsjosmqSg2J8Tq5xP2s2daDS6Ni4
240 : hashes per tick: 57,500
241 : hashcnt duration: 108.695652174 nanos
242 :
243 : name: update_hashes_per_tick6
244 : id: FKu1qYwLQSiehz644H6Si65U5ZQ2cp9GxsyFUfYcuADv
245 : hashes per tick: 62,500
246 : hashcnt duration: 100 nanos
247 :
248 : In development environments, there is a way to configure the
249 : hashcnt per tick to be "none" during genesis, for a so-called
250 : "low power" tick producer. The idea is not to spin cores during
251 : development. This is equivalent to setting the hashcnt per tick
252 : to be 1, and increasing the hashcnt duration to the desired tick
253 : duration.
254 :
255 : SLOTS
256 :
257 : Each leader needs to be leader for a fixed amount of time, which
258 : is called a slot. During a slot, a leader has an opportunity to
259 : receive transactions and produce a block for the network,
260 : although they may miss ("skip") the slot if they are offline or
261 : not behaving.
262 :
263 : In mainnet-beta, testnet, and devnet a slot is 64 ticks, or
264 : 4,000,000 hashcnts, or approximately 400 milliseconds.
265 :
266 : Due to the way the leader schedule is constructed, each leader
267 : is always given at least four (4) consecutive slots in the
268 : schedule. This means when becoming leader you will be leader
269 : for at least 4 slots, or 1.6 seconds.
270 :
271 : It is rare, although can happen that a leader gets more than 4
272 : consecutive slots (eg, 8, or 12), if they are lucky with the
273 : leader schedule generation.
274 :
275 : The number of ticks in a slot is fixed at genesis time, and
276 : could be different for development or other chains, which we
277 : support. There is nothing special about 4 leader slots in a
278 : row, and this might be changed in future, and the proof of
279 : history makes no assumptions that this is the case.
280 :
281 : EPOCHS
282 :
283 : Infrequently, the network needs to do certain housekeeping,
284 : mainly things like collecting rent and deciding on the leader
285 : schedule. The length of an epoch is fixed on mainnet-beta,
286 : devnet and testnet at 420,000 slots, or around ~2 (1.94) days.
287 : This value is fixed at genesis time, and could be different for
288 : other chains including development, which we support. Typically
289 : in development, epochs are every 8,192 slots, or around ~1 hour
290 : (54.61 minutes), although it depends on the number of ticks per
291 : slot and the target hashcnt rate of the genesis as well.
292 :
293 : In development, epochs need not be a fixed length either. There
294 : is a "warmup" option, where epochs start short and grow, which
295 : is useful for quickly warming up stake during development.
296 :
297 : The epoch is important because it is the only time the leader
298 : schedule is updated. The leader schedule is a list of which
299 : leader is leader for which slot, and is generated by a special
300 : algorithm that is deterministic and known to all nodes.
301 :
302 : The leader schedule is computed one epoch in advance, so that
303 : at slot T, we always know who will be leader up until the end
304 : of slot T+EPOCH_LENGTH. Specifically, the leader schedule for
305 : epoch N is computed during the epoch boundary crossing from
306 : N-2 to N-1. For mainnet-beta, the slots per epoch is fixed and
307 : will always be 420,000. */
308 :
309 : #include "../../disco/tiles.h"
310 : #include "../../disco/fd_txn_m.h"
311 : #include "../../disco/bundle/fd_bundle_crank.h"
312 : #include "../../disco/pack/fd_pack.h"
313 : #include "../../disco/pack/fd_pack_cost.h"
314 : #include "../../ballet/sha256/fd_sha256.h"
315 : #include "../../disco/metrics/fd_metrics.h"
316 : #include "../../util/pod/fd_pod.h"
317 : #include "../../disco/shred/fd_shredder.h"
318 : #include "../../disco/keyguard/fd_keyload.h"
319 : #include "../../disco/keyguard/fd_keyswitch.h"
320 : #include "../plugin/fd_plugin.h"
321 : #include "../../flamenco/leaders/fd_multi_epoch_leaders.h"
322 :
323 : #include <string.h>
324 :
325 : /* The maximum number of microblocks that pack is allowed to pack into a
326 : single slot. This is not consensus critical, and pack could, if we
327 : let it, produce as many microblocks as it wants, and the slot would
328 : still be valid.
329 :
330 : We have this here instead so that PoH can estimate slot completion,
331 : and keep the hashcnt up to date as pack progresses through packing
332 : the slot. If this upper bound was not enforced, PoH could tick to
333 : the last hash of the slot and have no hashes left to mixin incoming
334 : microblocks from pack, so this upper bound is a coordination
335 : mechanism so that PoH can progress hashcnts while the slot is active,
336 : and know that pack will not need those hashcnts later to do mixins. */
337 0 : #define MAX_MICROBLOCKS_PER_SLOT (131072UL)
338 :
339 : /* When we are hashing in the background in case a prior leader skips
340 : their slot, we need to store the result of each tick hash so we can
341 : publish them when we become leader. The network requires at least
342 : one leader slot to publish in each epoch for the leader schedule to
343 : generate, so in the worst case we might need two full epochs of slots
344 : to store the hashes. (Eg, if epoch T only had a published slot in
345 : position 0 and epoch T+1 only had a published slot right at the end).
346 :
347 : There is a tighter bound: the block data limit of mainnet-beta is
348 : currently FD_PACK_MAX_DATA_PER_BLOCK, or 27,332,342 bytes per slot.
349 : At 48 bytes per tick, it is not possible to publish a slot that skips
350 : 569,424 or more prior slots. */
351 0 : #define MAX_SKIPPED_TICKS (1UL+(FD_PACK_MAX_DATA_PER_BLOCK/48UL))
352 :
353 0 : #define IN_KIND_BANK (0)
354 0 : #define IN_KIND_PACK (1)
355 0 : #define IN_KIND_EPOCH (2)
356 :
357 :
358 : struct fd_pohh_in {
359 : fd_wksp_t * mem;
360 : ulong chunk0;
361 : ulong wmark;
362 : };
363 :
364 : typedef struct fd_pohh_in fd_pohh_in_t;
365 :
366 : struct fd_pohh_out {
367 : ulong idx;
368 : fd_wksp_t * mem;
369 : ulong chunk0;
370 : ulong wmark;
371 : ulong chunk;
372 : };
373 :
374 : typedef struct fd_pohh_out fd_pohh_out_t;
375 :
376 : struct fd_pohh_tile {
377 : fd_stem_context_t * stem;
378 :
379 : /* Static configuration determined at genesis creation time. See
380 : long comment above for more information. */
381 : ulong tick_duration_ns;
382 : ulong hashcnt_per_tick;
383 : ulong ticks_per_slot;
384 :
385 : /* Derived from the above configuration, but we precompute it. */
386 : double slot_duration_ns;
387 : double hashcnt_duration_ns;
388 : ulong hashcnt_per_slot;
389 :
390 : /* The maximum number of real microblocks that the pack tile is
391 : allowed to publish in each slot.
392 :
393 : While we are leader, PoH internally treats this limit as having
394 : one extra phantom "microblock" reserved for the done_packing
395 : message, so that PoH does not finish the slot before pack
396 : confirms it is done. Pack itself is configured with the
397 : un-inflated limit and never publishes more than this many real
398 : microblocks per slot. */
399 : ulong max_microblocks_per_slot;
400 :
401 : /* Consensus-critical slot cost limits. */
402 : struct {
403 : ulong slot_max_cost;
404 : ulong slot_max_vote_cost;
405 : ulong slot_max_write_cost_per_acct;
406 : ulong slot_max_allocated_data_per_block;
407 : ulong slot_max_data_shreds;
408 : } limits;
409 :
410 : /* The current slot and hashcnt within that slot of the proof of
411 : history, including hashes we have been producing in the background
412 : while waiting for our next leader slot. */
413 : ulong slot;
414 : ulong hashcnt;
415 : ulong cus_used;
416 :
417 : /* When we send a microblock on to the shred tile, we need to tell
418 : it how many hashes there have been since the last microblock, so
419 : this tracks the hashcnt of the last published microblock.
420 :
421 : If we are skipping slots prior to our leader slot, the last_slot
422 : will be quite old, and potentially much larger than the number of
423 : hashcnts in one slot. */
424 : ulong last_slot;
425 : ulong last_hashcnt;
426 :
427 : /* If we have published a tick or a microblock for a particular slot
428 : to the shred tile, we should never become leader for that slot
429 : again, otherwise we could publish a duplicate block.
430 :
431 : This value tracks the max slot that we have published a tick or
432 : microblock for so we can prevent this. */
433 : ulong highwater_leader_slot;
434 :
435 : /* See how this field is used below. If we have sequential leader
436 : slots, we don't reset the expected slot end time between the two,
437 : to prevent clock drift. If we didn't do this, our 2nd slot would
438 : end 400ms + `time_for_replay_to_move_slot_and_reset_poh` after
439 : our 1st, rather than just strictly 400ms. */
440 : int lagged_consecutive_leader_start;
441 : ulong expect_sequential_leader_slot;
442 :
443 : /* There's a race condition ... let's say two banks A and B, bank A
444 : processes some transactions, then releases the account locks, and
445 : sends the microblock to PoH to be stamped. Pack now re-packs the
446 : same accounts with a new microblock, sends to bank B, bank B
447 : executes and sends the microblock to PoH, and this all happens fast
448 : enough that PoH picks the 2nd block to stamp before the 1st. The
449 : accounts database changes now are misordered with respect to PoH so
450 : replay could fail.
451 :
452 : To prevent this race, we order all microblocks and only process
453 : them in PoH in the order they are produced by pack. This is a
454 : little bit over-strict, we just need to ensure that microblocks
455 : with conflicting accounts execute in order, but this is easiest to
456 : implement for now. */
457 : uint expect_pack_idx;
458 :
459 : /* Pack and bank tiles need a reference to the bank object with a
460 : slightly different lifetime than current_leader_bank, particularly
461 : when we switch forks in the middle of a leader slot. We need to
462 : make sure we don't free the last reference to the bank while the
463 : pack or bank tiles are still using it. The strange thing is that
464 : bank tiles have no concept of the current slot, but we know they're
465 : done with the bank object when pack's inter-slot bank draining
466 : process is complete. Pack notifies PoH by a frag with
467 : sig==ULONG_MAX on the pack_poh link when the banks are drained, and
468 : the PoH tile must then free the reference on behalf of pack.
469 :
470 : pack_leader_bank is non-NULL when the reference we're holding on
471 : behalf of the pack tile is acquired, and NULL when it is not
472 : acquired. */
473 : void const * pack_leader_bank;
474 :
475 : /* Store tile needs another reference to the bank object with its
476 : own lifetime requirements. We need to set the block_id of a
477 : slot to the merkle root of the last FEC set in the slot. We
478 : also want to make sure that we correctly release the reference
479 : in case we abandon a slot midway and never send a SLOT_COMPLETE.
480 : For the latter part, we track the slot we acquired the bank for
481 : so that we can signal to release it if we have moved on from
482 : that slot. Since poh does not have a direct link to store,
483 : we pass the bank pointer using the poh_shred -> shred_store
484 : route.*/
485 : void const * store_leader_bank;
486 : ulong store_leader_bank_slot;
487 :
488 : /* The PoH tile must never drop microblocks that get committed by the
489 : bank, so it needs to always be able to mixin a microblock hash.
490 : Mixing in requires incrementing the hashcnt, so we need to ensure
491 : at all times that there is enough hascnts left in the slot to
492 : mixin whatever future microblocks pack might produce for it.
493 :
494 : This value tracks that. At any time, max_microblocks_per_slot
495 : - microblocks_lower_bound is an upper bound on the maximum number
496 : of microblocks that might still be received in this slot. */
497 : ulong microblocks_lower_bound;
498 :
499 : uchar __attribute__((aligned(32UL))) reset_hash[ 32 ];
500 : uchar __attribute__((aligned(32UL))) hash[ 32 ];
501 :
502 : /* When we are not leader, we need to save the hashes that were
503 : produced in case the prior leader skips. If they skip, we will
504 : replay these skipped hashes into our next leader bank so that
505 : the slot hashes sysvar can be updated correctly, and also publish
506 : them to peer nodes as part of our outgoing shreds. */
507 : uchar skipped_tick_hashes[ MAX_SKIPPED_TICKS ][ 32 ];
508 :
509 : /* The timestamp in nanoseconds of when the reset slot was received.
510 : This is the timestamp we are building on top of to determine when
511 : our next leader slot starts. */
512 : long reset_slot_start_ns;
513 :
514 : /* The timestamp in nanoseconds of when we got the bank for the
515 : current leader slot. */
516 : long leader_bank_start_ns;
517 :
518 : /* The hashcnt corresponding to the start of the current reset slot. */
519 : ulong reset_slot;
520 :
521 : /* The hashcnt at which our next leader slot begins, or ULONG max if
522 : we have no known next leader slot. */
523 : ulong next_leader_slot;
524 :
525 : /* If an in progress frag should be skipped */
526 : int skip_frag;
527 :
528 : ulong max_active_descendant;
529 :
530 : /* If we currently are the leader according the clock AND we have
531 : received the leader bank for the slot from the replay stage,
532 : this value will be non-NULL.
533 :
534 : Note that we might be inside our leader slot, but not have a bank
535 : yet, in which case this will still be NULL.
536 :
537 : It will be NULL for a brief race period between consecutive leader
538 : slots, as we ping-pong back to replay stage waiting for a new bank.
539 :
540 : Agave refers to this as the "working bank". */
541 : void const * current_leader_bank;
542 :
543 : fd_sha256_t * sha256;
544 :
545 : fd_multi_epoch_leaders_t * mleaders;
546 :
547 : /* The last sequence number of an outgoing fragment to the shred tile,
548 : or ULONG max if no such fragment. See fd_keyswitch.h for details
549 : of how this is used. */
550 : ulong shred_seq;
551 :
552 : int halted_switching_key;
553 :
554 : fd_keyswitch_t * keyswitch;
555 : fd_pubkey_t identity_key;
556 :
557 : /* We need a few pieces of information to compute the right addresses
558 : for bundle crank information that we need to send to pack. */
559 : struct {
560 : int enabled;
561 : fd_pubkey_t vote_account;
562 : fd_bundle_crank_gen_t gen[1];
563 : } bundle;
564 :
565 :
566 : /* The Agave client needs to be notified when the leader changes,
567 : so that they can resume the replay stage if it was suspended waiting. */
568 : void * signal_leader_change;
569 :
570 : /* These are temporarily set in during_frag so they can be used in
571 : after_frag once the frag has been validated as not overrun. */
572 : uchar _txns[ USHORT_MAX ];
573 : fd_microblock_trailer_t _microblock_trailer[ 1 ];
574 :
575 : int in_kind[ 64 ];
576 : fd_pohh_in_t in[ 64 ];
577 :
578 : fd_pohh_out_t shred_out[ 1 ];
579 : fd_pohh_out_t pack_out[ 1 ];
580 : fd_pohh_out_t plugin_out[ 1 ];
581 :
582 : fd_histf_t begin_leader_delay[ 1 ];
583 : fd_histf_t first_microblock_delay[ 1 ];
584 : fd_histf_t slot_done_delay[ 1 ];
585 : fd_histf_t bundle_init_delay[ 1 ];
586 :
587 : ulong shred_epoch_msg_avail;
588 : fd_shred_epoch_msg_t shred_epoch_msg[1];
589 :
590 : ulong parent_slot;
591 : uchar parent_block_id[ 32 ];
592 :
593 : uchar __attribute__((aligned(FD_MULTI_EPOCH_LEADERS_ALIGN))) mleaders_mem[ FD_MULTI_EPOCH_LEADERS_FOOTPRINT ];
594 : };
595 :
596 : typedef struct fd_pohh_tile fd_pohh_tile_t;
597 :
598 : /* The PoH recorder is implemented in Firedancer but for now needs to
599 : work with Agave, so we have a locking scheme for them to
600 : co-operate.
601 :
602 : This is because the PoH tile lives in the Agave memory address
603 : space and their version of concurrency is locking the PoH recorder
604 : and reading arbitrary fields.
605 :
606 : So we allow them to lock the PoH tile, although with a very bad (for
607 : them) locking scheme. By default, the tile has full and exclusive
608 : access to the data. If part of Agave wishes to read/write they
609 : can either,
610 :
611 : 1. Rewrite their concurrency to message passing based on mcache
612 : (preferred, but not feasible).
613 : 2. Signal to the tile they wish to acquire the lock, by setting
614 : fd_poh_waiting_lock to 1.
615 :
616 : During after_credit, the tile will check if the waiting lock is set
617 : to 1, and if so, set the returned lock to 1, indicating to the waiter
618 : that they may now proceed.
619 :
620 : When the waiter is done reading and writing, they restore the
621 : returned lock value back to zero, and the POH tile continues with its
622 : day. */
623 :
624 : static fd_pohh_tile_t * fd_pohh_global_ctx;
625 :
626 : static volatile ulong fd_poh_waiting_lock __attribute__((aligned(128UL)));
627 : static volatile ulong fd_poh_returned_lock __attribute__((aligned(128UL)));
628 :
629 : /* Agave also needs to write to some mcaches, so we trampoline
630 : that via. the PoH tile as well. */
631 :
632 : struct poh_link {
633 : fd_frag_meta_t * mcache;
634 : ulong depth;
635 : ulong tx_seq;
636 :
637 : void * mem;
638 : void * dcache;
639 : ulong chunk0;
640 : ulong wmark;
641 : ulong chunk;
642 :
643 : ulong cr_avail;
644 : ulong rx_cnt;
645 : ulong * rx_fseqs[ 32UL ];
646 : };
647 :
648 : typedef struct poh_link poh_link_t;
649 :
650 : static poh_link_t gossip_dedup;
651 : static poh_link_t stake_out;
652 : static poh_link_t crds_shred;
653 : static poh_link_t replay_resolh;
654 : static poh_link_t executed_txn;
655 :
656 : static poh_link_t replay_plugin;
657 : static poh_link_t gossip_plugin;
658 : static poh_link_t start_progress_plugin;
659 : static poh_link_t vote_listener_plugin;
660 : static poh_link_t validator_info_plugin;
661 :
662 : static void
663 0 : poh_link_wait_credit( poh_link_t * link ) {
664 0 : if( FD_LIKELY( link->cr_avail ) ) return;
665 :
666 0 : while( 1 ) {
667 0 : ulong cr_query = ULONG_MAX;
668 0 : for( ulong i=0UL; i<link->rx_cnt; i++ ) {
669 0 : ulong const * _rx_seq = link->rx_fseqs[ i ];
670 0 : ulong rx_seq = FD_VOLATILE_CONST( *_rx_seq );
671 0 : ulong rx_cr_query = (ulong)fd_long_max( (long)link->depth - fd_long_max( fd_seq_diff( link->tx_seq, rx_seq ), 0L ), 0L );
672 0 : cr_query = fd_ulong_min( rx_cr_query, cr_query );
673 0 : }
674 0 : if( FD_LIKELY( cr_query>0UL ) ) {
675 0 : link->cr_avail = cr_query;
676 0 : break;
677 0 : }
678 0 : FD_SPIN_PAUSE();
679 0 : }
680 0 : }
681 :
682 : static void
683 : poh_link_publish( poh_link_t * link,
684 : ulong sig,
685 : uchar const * data,
686 0 : ulong data_sz ) {
687 0 : while( FD_UNLIKELY( !FD_VOLATILE_CONST( link->mcache ) ) ) FD_SPIN_PAUSE();
688 0 : if( FD_UNLIKELY( !link->mem ) ) return; /* link not enabled, don't publish */
689 0 : poh_link_wait_credit( link );
690 :
691 0 : uchar * dst = (uchar *)fd_chunk_to_laddr( link->mem, link->chunk );
692 0 : fd_memcpy( dst, data, data_sz );
693 0 : ulong tspub = (ulong)fd_frag_meta_ts_comp( fd_tickcount() );
694 0 : fd_mcache_publish( link->mcache, link->depth, link->tx_seq, sig, link->chunk, data_sz, 0UL, 0UL, tspub );
695 0 : link->chunk = fd_dcache_compact_next( link->chunk, data_sz, link->chunk0, link->wmark );
696 0 : link->cr_avail--;
697 0 : link->tx_seq++;
698 0 : }
699 :
700 : static void
701 : poh_link_init( poh_link_t * link,
702 : fd_topo_t const * topo,
703 : fd_topo_tile_t const * tile,
704 0 : ulong out_idx ) {
705 0 : fd_topo_link_t const * topo_link = &topo->links[ tile->out_link_id[ out_idx ] ];
706 0 : fd_topo_wksp_t const * wksp = &topo->workspaces[ topo->objs[ topo_link->dcache_obj_id ].wksp_id ];
707 :
708 0 : link->mem = wksp->wksp;
709 0 : link->depth = fd_mcache_depth( topo_link->mcache );
710 0 : link->tx_seq = 0UL;
711 0 : link->dcache = topo_link->dcache;
712 0 : link->chunk0 = fd_dcache_compact_chunk0( wksp->wksp, topo_link->dcache );
713 0 : link->wmark = fd_dcache_compact_wmark ( wksp->wksp, topo_link->dcache, topo_link->mtu );
714 0 : link->chunk = link->chunk0;
715 0 : link->cr_avail = 0UL;
716 0 : link->rx_cnt = 0UL;
717 0 : for( ulong i=0UL; i<topo->tile_cnt; i++ ) {
718 0 : fd_topo_tile_t const * _tile = &topo->tiles[ i ];
719 0 : for( ulong j=0UL; j<_tile->in_cnt; j++ ) {
720 0 : if( _tile->in_link_id[ j ]==topo_link->id && _tile->in_link_reliable[ j ] ) {
721 0 : FD_TEST( link->rx_cnt<32UL );
722 0 : link->rx_fseqs[ link->rx_cnt++ ] = _tile->in_link_fseq[ j ];
723 0 : break;
724 0 : }
725 0 : }
726 0 : }
727 0 : FD_COMPILER_MFENCE();
728 0 : link->mcache = topo_link->mcache;
729 0 : FD_COMPILER_MFENCE();
730 0 : FD_TEST( link->mcache );
731 0 : }
732 :
733 : /* To help show correctness, functions that might be called from
734 : Rust, either directly or indirectly, have this fake "attribute"
735 : CALLED_FROM_RUST, which is actually nothing. Calls from Rust
736 : typically execute on threads did not call fd_boot, so they do not
737 : have the typical FD_TL variables. In particular, they cannot use
738 : normal metrics, and their log messages don't have full context.
739 : Additionally, Rust functions marked CALLED_FROM_RUST cannot call back
740 : into a C fd_ext function without causing a deadlock (although the
741 : other Rust fd_ext functions have a similar problem).
742 :
743 : To prevent annotation from polluting the whole codebase, calls to
744 : functions outside this file are manually checked and marked as being
745 : safe at each call rather than annotated. */
746 : #define CALLED_FROM_RUST
747 :
748 : static CALLED_FROM_RUST fd_pohh_tile_t *
749 0 : fd_ext_poh_write_lock( void ) {
750 0 : for(;;) {
751 : /* Acquire the waiter lock to make sure we are the first writer in the queue. */
752 0 : if( FD_LIKELY( !FD_ATOMIC_CAS( &fd_poh_waiting_lock, 0UL, 1UL) ) ) break;
753 0 : FD_SPIN_PAUSE();
754 0 : }
755 0 : FD_COMPILER_MFENCE();
756 0 : for(;;) {
757 : /* Now wait for the tile to tell us we can proceed. */
758 0 : if( FD_LIKELY( FD_VOLATILE_CONST( fd_poh_returned_lock ) ) ) break;
759 0 : FD_SPIN_PAUSE();
760 0 : }
761 0 : FD_COMPILER_MFENCE();
762 0 : return fd_pohh_global_ctx;
763 0 : }
764 :
765 : static CALLED_FROM_RUST void
766 0 : fd_ext_poh_write_unlock( void ) {
767 0 : FD_COMPILER_MFENCE();
768 0 : FD_VOLATILE( fd_poh_returned_lock ) = 0UL;
769 0 : }
770 :
771 : /* The PoH tile needs to interact with the Agave address space to
772 : do certain operations that Firedancer hasn't reimplemented yet, a.k.a
773 : transaction execution. We have Agave export some wrapper
774 : functions that we call into during regular tile execution. These do
775 : not need any locking, since they are called serially from the single
776 : PoH tile. */
777 :
778 : extern CALLED_FROM_RUST void fd_ext_bank_acquire( void const * bank );
779 : extern CALLED_FROM_RUST void fd_ext_bank_release( void const * bank );
780 : extern CALLED_FROM_RUST void fd_ext_poh_signal_leader_change( void * sender );
781 : extern void fd_ext_poh_register_tick( void const * bank, uchar const * hash );
782 :
783 : /* fd_ext_poh_initialize is called by Agave on startup to
784 : initialize the PoH tile with some static configuration, and the
785 : initial reset slot and hash which it retrieves from a snapshot.
786 :
787 : This function is called by some random Agave thread, but
788 : it blocks booting of the PoH tile. The tile will spin until it
789 : determines that this initialization has happened.
790 :
791 : signal_leader_change is an opaque Rust object that is used to
792 : tell the replay stage that the leader has changed. It is a
793 : Box::into_raw(Arc::increment_strong(crossbeam::Sender)), so it
794 : has infinite lifetime unless this C code releases the refcnt.
795 :
796 : It can be used with `fd_ext_poh_signal_leader_change` which
797 : will just issue a nonblocking send on the channel. */
798 :
799 : CALLED_FROM_RUST void
800 : fd_ext_poh_initialize( ulong tick_duration_ns, /* See clock comments above, will be 6.4 microseconds for mainnet-beta. */
801 : ulong hashcnt_per_tick, /* See clock comments above, will be 62,500 for mainnet-beta. */
802 : ulong ticks_per_slot, /* See clock comments above, will almost always be 64. */
803 : ulong tick_height, /* The counter (height) of the tick to start hashing on top of. */
804 : uchar const * last_entry_hash, /* Points to start of a 32 byte region of memory, the hash itself at the tick height. */
805 0 : void * signal_leader_change /* See comment above. */ ) {
806 0 : FD_COMPILER_MFENCE();
807 0 : for(;;) {
808 : /* Make sure the ctx is initialized before trying to take the lock. */
809 0 : if( FD_LIKELY( FD_VOLATILE_CONST( fd_pohh_global_ctx ) ) ) break;
810 0 : FD_SPIN_PAUSE();
811 0 : }
812 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
813 :
814 0 : ctx->slot = tick_height/ticks_per_slot;
815 0 : ctx->hashcnt = 0UL;
816 0 : ctx->cus_used = 0UL;
817 0 : ctx->last_slot = ctx->slot;
818 0 : ctx->last_hashcnt = 0UL;
819 0 : ctx->reset_slot = ctx->slot;
820 0 : ctx->reset_slot_start_ns = fd_log_wallclock(); /* safe to call from Rust */
821 :
822 0 : memcpy( ctx->reset_hash, last_entry_hash, 32UL );
823 0 : memcpy( ctx->hash, last_entry_hash, 32UL );
824 :
825 0 : ctx->signal_leader_change = signal_leader_change;
826 :
827 : /* Static configuration about the clock. */
828 0 : ctx->tick_duration_ns = tick_duration_ns;
829 0 : ctx->hashcnt_per_tick = hashcnt_per_tick;
830 0 : ctx->ticks_per_slot = ticks_per_slot;
831 :
832 : /* Recompute derived information about the clock. */
833 0 : ctx->slot_duration_ns = (double)ticks_per_slot*(double)tick_duration_ns;
834 0 : ctx->hashcnt_duration_ns = (double)tick_duration_ns/(double)hashcnt_per_tick;
835 0 : ctx->hashcnt_per_slot = ticks_per_slot*hashcnt_per_tick;
836 :
837 0 : if( FD_UNLIKELY( ctx->hashcnt_per_tick==1UL ) ) {
838 : /* Low power producer, maximum of one microblock per tick in the slot */
839 0 : ctx->max_microblocks_per_slot = ctx->ticks_per_slot;
840 0 : } else {
841 : /* See the long comment in after_credit for this limit */
842 0 : ctx->max_microblocks_per_slot = fd_ulong_min( MAX_MICROBLOCKS_PER_SLOT, ctx->ticks_per_slot*(ctx->hashcnt_per_tick-1UL) );
843 0 : }
844 :
845 0 : fd_ext_poh_write_unlock();
846 0 : }
847 :
848 : /* fd_ext_poh_acquire_bank gets the current leader bank if there is one
849 : currently active. PoH might think we are leader without having a
850 : leader bank if the replay stage has not yet noticed we are leader.
851 :
852 : The bank that is returned is owned the caller, and must be converted
853 : to an Arc<Bank> by calling Arc::from_raw() on it. PoH increments the
854 : reference count before returning the bank, so that it can also keep
855 : its internal copy.
856 :
857 : If there is no leader bank, NULL is returned. In this case, the
858 : caller should not call `Arc::from_raw()`. */
859 :
860 : CALLED_FROM_RUST void const *
861 0 : fd_ext_poh_acquire_leader_bank( void ) {
862 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
863 0 : void const * bank = NULL;
864 0 : if( FD_LIKELY( ctx->current_leader_bank ) ) {
865 : /* Clone refcount before we release the lock. */
866 0 : fd_ext_bank_acquire( ctx->current_leader_bank );
867 0 : bank = ctx->current_leader_bank;
868 0 : }
869 0 : fd_ext_poh_write_unlock();
870 0 : return bank;
871 0 : }
872 :
873 : /* fd_ext_poh_reset_slot returns the slot height one above the last good
874 : (unskipped) slot we are building on top of. This is always a good
875 : known value, and will not be ULONG_MAX. */
876 :
877 : CALLED_FROM_RUST ulong
878 0 : fd_ext_poh_reset_slot( void ) {
879 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
880 0 : ulong reset_slot = ctx->reset_slot;
881 0 : fd_ext_poh_write_unlock();
882 0 : return reset_slot;
883 0 : }
884 :
885 : CALLED_FROM_RUST void
886 0 : fd_ext_poh_update_active_descendant( ulong max_active_descendant ) {
887 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
888 0 : ctx->max_active_descendant = max_active_descendant;
889 0 : fd_ext_poh_write_unlock();
890 0 : }
891 :
892 : /* fd_ext_poh_reached_leader_slot returns 1 if we have reached a slot
893 : where we are leader. This is used by the replay stage to determine
894 : if it should create a new leader bank descendant of the prior reset
895 : slot block.
896 :
897 : Sometimes, even when we reach our slot we do not return 1, as we are
898 : giving a grace period to the prior leader to finish publishing their
899 : block.
900 :
901 : out_leader_slot is the slot height of the leader slot we reached, and
902 : reset_slot is the slot height of the last good (unskipped) slot we
903 : are building on top of. */
904 :
905 : CALLED_FROM_RUST int
906 : fd_ext_poh_reached_leader_slot( ulong * out_leader_slot,
907 0 : ulong * out_reset_slot ) {
908 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
909 :
910 0 : *out_leader_slot = ctx->next_leader_slot;
911 0 : *out_reset_slot = ctx->reset_slot;
912 :
913 0 : if( FD_UNLIKELY( ctx->next_leader_slot==ULONG_MAX ||
914 0 : ctx->slot<ctx->next_leader_slot ) ) {
915 : /* Didn't reach our leader slot yet. */
916 0 : fd_ext_poh_write_unlock();
917 0 : return 0;
918 0 : }
919 :
920 0 : if( FD_UNLIKELY( ctx->halted_switching_key ) ) {
921 : /* Reached our leader slot, but the leader pipeline is halted
922 : because we are switching identity key. */
923 0 : fd_ext_poh_write_unlock();
924 0 : return 0;
925 0 : }
926 :
927 0 : if( FD_LIKELY( ctx->reset_slot==ctx->next_leader_slot ) ) {
928 : /* We were reset onto our leader slot, because the prior leader
929 : completed theirs, so we should start immediately, no need for a
930 : grace period. */
931 0 : fd_ext_poh_write_unlock();
932 0 : return 1;
933 0 : }
934 :
935 0 : fd_pubkey_t const * reset_leader = fd_multi_epoch_leaders_get_leader_for_slot( ctx->mleaders, ctx->reset_slot );
936 0 : if( FD_UNLIKELY( reset_leader && fd_memeq( reset_leader, ctx->identity_key.uc, 32UL ) ) ) {
937 : /* Surprisingly, in some rare cases where we're skipping ourselves,
938 : the following can occur:
939 : Reset onto n-1
940 : Tick into slot n, become leader for slot n, skipping slot n-1
941 : Prior leader start publishing slot n-1
942 : max_active_descendant is set to n
943 : Switch forks, abandon slot n, reset onto slot n
944 : In this case, next_leader_slot is n+1 because we can't become
945 : leader again for slot n. We don't want to give ourselves any
946 : grace time though; we want to start n+1 as soon as the hashing
947 : is ready. */
948 0 : fd_ext_poh_write_unlock();
949 0 : return 1;
950 0 : }
951 :
952 0 : long now_ns = fd_log_wallclock();
953 0 : long expected_start_time_ns = ctx->reset_slot_start_ns + (long)((double)(ctx->next_leader_slot-ctx->reset_slot)*ctx->slot_duration_ns);
954 :
955 : /* Now we're faced with the question of how much grace to give the
956 : prior leader before trying to skip them. If they are still in the
957 : process of publishing their slot, delay ours to let them finish ...
958 : unless they are so delayed that we risk getting skipped by the
959 : leader following us. 1.2 seconds is a reasonable default here,
960 : although any value between 0 and 1.6 seconds could be considered
961 : reasonable. If they haven't started their last block, but we're
962 : reset on their second to last block, we'll give them an extra
963 : 400ms. This is arbitrary and chosen due to intuition. */
964 :
965 0 : long start_time_with_grace_ns = expected_start_time_ns;
966 :
967 0 : if( FD_UNLIKELY( ctx->max_active_descendant>=ctx->next_leader_slot ) ) {
968 : /* If the max_active_descendant is >= next_leader_slot, we waited
969 : too long and a leader after us started publishing to try and skip
970 : us. Just start our leader slot immediately, we might win ... */
971 0 : start_time_with_grace_ns = now_ns;
972 0 : } else if( FD_LIKELY( ctx->max_active_descendant>=ctx->reset_slot ) ) {
973 : /* If one of the leaders between the reset slot and our leader
974 : slot is in the process of publishing (they have a descendant
975 : bank that is in progress of being replayed), then keep waiting.
976 : We probably wouldn't get a leader slot out before they
977 : finished. */
978 0 : start_time_with_grace_ns += (long)(3.0*ctx->slot_duration_ns);
979 0 : } else if( FD_LIKELY( ctx->next_leader_slot==ctx->reset_slot+1UL ) ) {
980 : /* We finished replaying the slot two before ours, which means the
981 : prior leader is probably online, but they haven't started
982 : publishing the slot immediately prior to ours. Give the prior
983 : leader a little more time. */
984 0 : start_time_with_grace_ns += (long)(1.0*ctx->slot_duration_ns);
985 0 : }
986 :
987 :
988 0 : if( FD_UNLIKELY( now_ns<start_time_with_grace_ns ) ) {
989 0 : fd_ext_poh_write_unlock();
990 0 : return 0;
991 0 : }
992 :
993 0 : fd_ext_poh_write_unlock();
994 0 : return 1;
995 0 : }
996 :
997 : CALLED_FROM_RUST static inline void
998 : publish_plugin_slot_start( fd_pohh_tile_t * ctx,
999 : ulong slot,
1000 0 : ulong parent_slot ) {
1001 0 : if( FD_UNLIKELY( !ctx->plugin_out->mem ) ) return;
1002 :
1003 0 : fd_plugin_msg_slot_start_t * slot_start = (fd_plugin_msg_slot_start_t *)fd_chunk_to_laddr( ctx->plugin_out->mem, ctx->plugin_out->chunk );
1004 0 : *slot_start = (fd_plugin_msg_slot_start_t){ .slot = slot, .parent_slot = parent_slot };
1005 0 : fd_stem_publish( ctx->stem, ctx->plugin_out->idx, FD_PLUGIN_MSG_SLOT_START, ctx->plugin_out->chunk, sizeof(fd_plugin_msg_slot_start_t), 0UL, 0UL, 0UL );
1006 0 : ctx->plugin_out->chunk = fd_dcache_compact_next( ctx->plugin_out->chunk, sizeof(fd_plugin_msg_slot_start_t), ctx->plugin_out->chunk0, ctx->plugin_out->wmark );
1007 0 : }
1008 :
1009 : CALLED_FROM_RUST static inline void
1010 : publish_plugin_slot_end( fd_pohh_tile_t * ctx,
1011 : ulong slot,
1012 0 : ulong cus_used ) {
1013 0 : if( FD_UNLIKELY( !ctx->plugin_out->mem ) ) return;
1014 :
1015 0 : fd_plugin_msg_slot_end_t * slot_end = (fd_plugin_msg_slot_end_t *)fd_chunk_to_laddr( ctx->plugin_out->mem, ctx->plugin_out->chunk );
1016 0 : *slot_end = (fd_plugin_msg_slot_end_t){ .slot = slot, .cus_used = cus_used };
1017 0 : fd_stem_publish( ctx->stem, ctx->plugin_out->idx, FD_PLUGIN_MSG_SLOT_END, ctx->plugin_out->chunk, sizeof(fd_plugin_msg_slot_end_t), 0UL, 0UL, 0UL );
1018 0 : ctx->plugin_out->chunk = fd_dcache_compact_next( ctx->plugin_out->chunk, sizeof(fd_plugin_msg_slot_end_t), ctx->plugin_out->chunk0, ctx->plugin_out->wmark );
1019 0 : }
1020 :
1021 : extern int
1022 : fd_ext_bank_load_account( void const * bank,
1023 : int fixed_root,
1024 : uchar const * addr,
1025 : uchar * owner,
1026 : uchar * data,
1027 : ulong * data_sz );
1028 :
1029 : CALLED_FROM_RUST static void
1030 : publish_became_leader( fd_pohh_tile_t * ctx,
1031 : ulong slot,
1032 0 : ulong epoch ) {
1033 0 : double tick_per_ns = fd_tempo_tick_per_ns( NULL );
1034 0 : fd_histf_sample( ctx->begin_leader_delay, (ulong)((double)(fd_log_wallclock()-ctx->reset_slot_start_ns)/tick_per_ns) );
1035 :
1036 0 : if( FD_UNLIKELY( ctx->lagged_consecutive_leader_start || ctx->reset_slot!=slot ) ) {
1037 : /* If we are mirroring Agave behavior, the wall clock gets reset
1038 : here so we don't count time spent waiting for a bank to freeze
1039 : or replay stage to actually start the slot towards our 400ms.
1040 :
1041 : See extended comments in the config file on this option.
1042 :
1043 : We must also reset the wall clock if we skipped a slot. */
1044 0 : ctx->reset_slot_start_ns = fd_log_wallclock() - (long)((double)(slot-ctx->reset_slot)*ctx->slot_duration_ns);
1045 0 : }
1046 :
1047 0 : fd_bundle_crank_tip_payment_config_t config[1] = { 0 };
1048 0 : fd_acct_addr_t tip_receiver_owner[1] = { 0 };
1049 :
1050 0 : if( FD_UNLIKELY( ctx->bundle.enabled ) ) {
1051 0 : long bundle_time = -fd_tickcount();
1052 0 : fd_acct_addr_t tip_payment_config[1];
1053 0 : fd_acct_addr_t tip_receiver[1];
1054 0 : fd_bundle_crank_get_addresses( ctx->bundle.gen, epoch, tip_payment_config, tip_receiver );
1055 :
1056 0 : fd_acct_addr_t _dummy[1];
1057 0 : uchar dummy[1];
1058 :
1059 0 : void const * bank = ctx->current_leader_bank;
1060 :
1061 : /* Calling rust from a C function that is CALLED_FROM_RUST risks
1062 : deadlock. In this case, I checked the load_account function and
1063 : ensured it never calls any C functions that acquire the lock. */
1064 0 : ulong sz1 = sizeof(config), sz2 = 1UL;
1065 0 : int found1 = fd_ext_bank_load_account( bank, 0, tip_payment_config->b, _dummy->b, (uchar *)config, &sz1 );
1066 0 : int found2 = fd_ext_bank_load_account( bank, 0, tip_receiver->b, tip_receiver_owner->b, dummy, &sz2 );
1067 : /* The bundle crank code detects whether the accounts were found by
1068 : whether they have non-zero values (since found and uninitialized
1069 : should be treated the same), so we actually don't really care
1070 : about the value of found{1,2}. */
1071 0 : (void)found1; (void)found2;
1072 0 : bundle_time += fd_tickcount();
1073 0 : fd_histf_sample( ctx->bundle_init_delay, (ulong)bundle_time );
1074 0 : }
1075 :
1076 0 : long slot_start_ns = ctx->reset_slot_start_ns + (long)((double)(slot-ctx->reset_slot)*ctx->slot_duration_ns);
1077 :
1078 : /* No need to check flow control, there are always credits became when we
1079 : are leader, we will not "become" leader again until we are done, so at
1080 : most one frag in flight at a time. */
1081 :
1082 0 : uchar * dst = (uchar *)fd_chunk_to_laddr( ctx->pack_out->mem, ctx->pack_out->chunk );
1083 :
1084 0 : fd_became_leader_t * leader = (fd_became_leader_t *)dst;
1085 0 : leader->slot_start_ns = slot_start_ns;
1086 0 : leader->slot_end_ns = (long)((double)slot_start_ns + ctx->slot_duration_ns);
1087 0 : leader->bank = ctx->current_leader_bank;
1088 0 : leader->max_microblocks_in_slot = ctx->max_microblocks_per_slot;
1089 0 : leader->ticks_per_slot = ctx->ticks_per_slot;
1090 0 : leader->tick_duration_ns = ctx->tick_duration_ns;
1091 0 : leader->hashcnt_per_tick = ctx->hashcnt_per_tick;
1092 0 : leader->total_skipped_ticks = ctx->ticks_per_slot*(slot-ctx->reset_slot);
1093 0 : leader->epoch = epoch;
1094 0 : leader->bundle->config[0] = config[0];
1095 0 : leader->slot = slot;
1096 :
1097 0 : leader->limits.slot_max_cost = ctx->limits.slot_max_cost;
1098 0 : leader->limits.slot_max_vote_cost = ctx->limits.slot_max_vote_cost;
1099 0 : leader->limits.slot_max_write_cost_per_acct = ctx->limits.slot_max_write_cost_per_acct;
1100 0 : leader->limits.slot_max_allocated_data_per_block = ctx->limits.slot_max_allocated_data_per_block;
1101 0 : leader->limits.slot_max_data_shreds = ctx->limits.slot_max_data_shreds;
1102 :
1103 0 : memcpy( leader->bundle->last_blockhash, ctx->reset_hash, 32UL );
1104 0 : memcpy( leader->bundle->tip_receiver_owner, tip_receiver_owner, 32UL );
1105 :
1106 0 : if( FD_UNLIKELY( leader->ticks_per_slot+leader->total_skipped_ticks>=MAX_SKIPPED_TICKS ) )
1107 0 : FD_LOG_ERR(( "Too many skipped ticks %lu for slot %lu, chain must halt", leader->ticks_per_slot+leader->total_skipped_ticks, slot ));
1108 :
1109 : /* increment refcount once for pack's reference to the current leader bank
1110 : and once for store's reference. */
1111 0 : if( FD_UNLIKELY( ctx->current_leader_bank ) ) {
1112 0 : ctx->pack_leader_bank = ctx->current_leader_bank;
1113 0 : fd_ext_bank_acquire( ctx->pack_leader_bank );
1114 :
1115 0 : FD_TEST( ctx->store_leader_bank_slot==ULONG_MAX );
1116 0 : ctx->store_leader_bank = ctx->current_leader_bank;
1117 0 : ctx->store_leader_bank_slot = slot;
1118 0 : fd_ext_bank_acquire( ctx->store_leader_bank );
1119 0 : }
1120 :
1121 0 : ulong pack_sig = fd_disco_poh_sig( slot, POH_PKT_TYPE_BECAME_LEADER, 0UL );
1122 0 : fd_stem_publish( ctx->stem, ctx->pack_out->idx, pack_sig, ctx->pack_out->chunk, sizeof(fd_became_leader_t), 0UL, 0UL, fd_frag_meta_ts_comp( fd_tickcount() ) );
1123 0 : ctx->pack_out->chunk = fd_dcache_compact_next( ctx->pack_out->chunk, sizeof(fd_became_leader_t), ctx->pack_out->chunk0, ctx->pack_out->wmark );
1124 :
1125 : /* We acquired another leader bank earlier that the store tile will use
1126 : to set the block_id for the slot we produce. We send this through
1127 : the existing poh_shred and shred_store links. */
1128 0 : void const ** msg = (void const **)fd_chunk_to_laddr( ctx->shred_out->mem, ctx->shred_out->chunk );
1129 0 : *msg = ctx->current_leader_bank;
1130 :
1131 0 : ulong shred_sig = fd_disco_poh_sig( slot, POH_PKT_TYPE_LEADER_BANK, 0UL );
1132 0 : fd_stem_publish( ctx->stem, ctx->shred_out->idx, shred_sig, ctx->shred_out->chunk, sizeof(void const *), 0UL, 0UL, fd_frag_meta_ts_comp( fd_tickcount() ) );
1133 0 : ctx->shred_seq = ctx->stem->seqs[ ctx->shred_out->idx ];
1134 0 : ctx->shred_out->chunk = fd_dcache_compact_next( ctx->shred_out->chunk, sizeof(void const *), ctx->shred_out->chunk0, ctx->shred_out->wmark );
1135 0 : }
1136 :
1137 : /* The PoH tile knows when it should become leader by waiting for its
1138 : leader slot (with the operating system clock). This function is so
1139 : that when it becomes the leader, it can be told what the leader bank
1140 : is by the replay stage. See the notes in the long comment above for
1141 : more on how this works. */
1142 :
1143 : CALLED_FROM_RUST void
1144 : fd_ext_poh_begin_leader( void const * bank,
1145 : ulong slot,
1146 : ulong epoch,
1147 : ulong hashcnt_per_tick,
1148 : ulong tick_duration_ns,
1149 : ulong cus_block_limit,
1150 : ulong cus_vote_cost_limit,
1151 : ulong cus_account_cost_limit,
1152 : ulong cus_allocated_data_size_limit,
1153 0 : ulong max_data_shreds ) {
1154 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
1155 :
1156 0 : FD_TEST( !ctx->current_leader_bank );
1157 :
1158 0 : if( FD_UNLIKELY( slot!=ctx->slot ) ) FD_LOG_ERR(( "Trying to begin leader slot %lu but we are now on slot %lu", slot, ctx->slot ));
1159 0 : if( FD_UNLIKELY( slot!=ctx->next_leader_slot ) ) FD_LOG_ERR(( "Trying to begin leader slot %lu but next leader slot is %lu", slot, ctx->next_leader_slot ));
1160 :
1161 0 : if( FD_UNLIKELY( ctx->tick_duration_ns!=tick_duration_ns ) ) {
1162 0 : FD_LOG_WARNING(( "tick duration changed from %lu to %lu", ctx->tick_duration_ns, tick_duration_ns ));
1163 :
1164 : /* Recompute derived information about the clock. */
1165 0 : ctx->tick_duration_ns = tick_duration_ns;
1166 0 : ctx->slot_duration_ns = (double)ctx->ticks_per_slot*(double)tick_duration_ns;
1167 0 : ctx->hashcnt_duration_ns = (double)tick_duration_ns/(double)hashcnt_per_tick;
1168 0 : }
1169 :
1170 0 : if( FD_UNLIKELY( ctx->hashcnt_per_tick!=hashcnt_per_tick ) ) {
1171 0 : FD_LOG_WARNING(( "hashes per tick changed from %lu to %lu", ctx->hashcnt_per_tick, hashcnt_per_tick ));
1172 :
1173 : /* Recompute derived information about the clock. */
1174 0 : ctx->hashcnt_duration_ns = (double)ctx->tick_duration_ns/(double)hashcnt_per_tick;
1175 0 : ctx->hashcnt_per_slot = ctx->ticks_per_slot*hashcnt_per_tick;
1176 0 : ctx->hashcnt_per_tick = hashcnt_per_tick;
1177 :
1178 : /* Discard any ticks we might have done in the interim. They will
1179 : have the wrong number of hashes per tick. We can just catch back
1180 : up quickly if not too many slots were skipped and hopefully
1181 : publish on time. Note that tick production and verification of
1182 : skipped slots is done for the eventual bank that publishes a
1183 : slot, for example:
1184 :
1185 : Reset Slot: 998
1186 : Epoch Transition Slot: 1000
1187 : Leader Slot: 1002
1188 :
1189 : In this case, if a feature changing the hashcnt_per_tick is
1190 : activated in slot 1000, and we are publishing empty ticks for
1191 : slots 998, 999, 1000, and 1001, they should all have the new
1192 : hashes_per_tick number of hashes, rather than the older one, or
1193 : some combination. */
1194 :
1195 0 : FD_TEST( ctx->last_slot==ctx->reset_slot );
1196 0 : FD_TEST( !ctx->last_hashcnt );
1197 0 : ctx->slot = ctx->reset_slot;
1198 0 : ctx->hashcnt = 0UL;
1199 0 : }
1200 :
1201 0 : if( FD_UNLIKELY( ctx->hashcnt_per_tick==1UL ) ) {
1202 : /* Low power producer, maximum of one microblock per tick in the slot */
1203 0 : ctx->max_microblocks_per_slot = ctx->ticks_per_slot;
1204 0 : } else {
1205 : /* See the long comment in after_credit for this limit */
1206 0 : ctx->max_microblocks_per_slot = fd_ulong_min( MAX_MICROBLOCKS_PER_SLOT, ctx->ticks_per_slot*(ctx->hashcnt_per_tick-1UL) );
1207 0 : }
1208 :
1209 0 : ctx->current_leader_bank = bank;
1210 0 : ctx->microblocks_lower_bound = 0UL;
1211 0 : ctx->cus_used = 0UL;
1212 :
1213 0 : ctx->limits.slot_max_cost = cus_block_limit;
1214 0 : ctx->limits.slot_max_vote_cost = cus_vote_cost_limit;
1215 0 : ctx->limits.slot_max_write_cost_per_acct = cus_account_cost_limit;
1216 0 : ctx->limits.slot_max_allocated_data_per_block = cus_allocated_data_size_limit;
1217 0 : ctx->limits.slot_max_data_shreds = max_data_shreds;
1218 :
1219 : /* clamp and warn if we are underutilizing CUs */
1220 0 : if( FD_UNLIKELY( ctx->limits.slot_max_cost > FD_PACK_MAX_COST_PER_BLOCK_UPPER_BOUND ) ) {
1221 0 : FD_LOG_WARNING(( "Underutilizing protocol slot CU limit. protocol_limit=%lu validator_limit=%lu", ctx->limits.slot_max_cost, FD_PACK_MAX_COST_PER_BLOCK_UPPER_BOUND ));
1222 0 : ctx->limits.slot_max_cost = FD_PACK_MAX_COST_PER_BLOCK_UPPER_BOUND;
1223 0 : }
1224 0 : if( FD_UNLIKELY( ctx->limits.slot_max_vote_cost > FD_PACK_MAX_VOTE_COST_PER_BLOCK_UPPER_BOUND ) ) {
1225 0 : FD_LOG_WARNING(( "Underutilizing protocol vote CU limit. protocol_limit=%lu validator_limit=%lu", ctx->limits.slot_max_vote_cost, FD_PACK_MAX_VOTE_COST_PER_BLOCK_UPPER_BOUND ));
1226 0 : ctx->limits.slot_max_vote_cost = FD_PACK_MAX_VOTE_COST_PER_BLOCK_UPPER_BOUND;
1227 0 : }
1228 0 : if( FD_UNLIKELY( ctx->limits.slot_max_write_cost_per_acct > FD_PACK_MAX_WRITE_COST_PER_ACCT_UPPER_BOUND ) ) {
1229 0 : FD_LOG_WARNING(( "Underutilizing protocol write CU limit. protocol_limit=%lu validator_limit=%lu", ctx->limits.slot_max_write_cost_per_acct, FD_PACK_MAX_WRITE_COST_PER_ACCT_UPPER_BOUND ));
1230 0 : ctx->limits.slot_max_write_cost_per_acct = FD_PACK_MAX_WRITE_COST_PER_ACCT_UPPER_BOUND;
1231 0 : }
1232 :
1233 : /* We are about to start publishing to the shred tile for this slot
1234 : so update the highwater mark so we never republish in this slot
1235 : again. Also check that the leader slot is greater than the
1236 : highwater, which should have been ensured earlier. */
1237 :
1238 0 : FD_TEST( ctx->highwater_leader_slot==ULONG_MAX || slot>=ctx->highwater_leader_slot );
1239 0 : ctx->highwater_leader_slot = fd_ulong_max( fd_ulong_if( ctx->highwater_leader_slot==ULONG_MAX, 0UL, ctx->highwater_leader_slot ), slot );
1240 :
1241 0 : publish_became_leader( ctx, slot, epoch );
1242 :
1243 : /* PoH ends the slot once it "ticks" through all of the hashes, but
1244 : we only want that to happen if we received a done packing message
1245 : from pack, so we always reserve an empty microblock at the end so
1246 : the tick advance will not end the slot without being told.
1247 :
1248 : This should be after publish_became_leader so that pack receives
1249 : the original (un-inflated) max_microblocks_per_slot. */
1250 0 : ctx->max_microblocks_per_slot += 1UL;
1251 :
1252 0 : FD_LOG_INFO(( "fd_ext_poh_begin_leader(slot=%lu, highwater_leader_slot=%lu, last_slot=%lu, last_hashcnt=%lu)", slot, ctx->highwater_leader_slot, ctx->last_slot, ctx->last_hashcnt ));
1253 :
1254 0 : fd_ext_poh_write_unlock();
1255 0 : }
1256 :
1257 : /* Determine what the next slot is in the leader schedule is that we are
1258 : leader. Includes the current slot. If we are not leader in what
1259 : remains of the current and next epoch, return ULONG_MAX. */
1260 :
1261 : static inline CALLED_FROM_RUST ulong
1262 0 : next_leader_slot( fd_pohh_tile_t * ctx ) {
1263 : /* If we have published anything in a particular slot, then we
1264 : should never become leader for that slot again. */
1265 0 : ulong min_leader_slot = fd_ulong_max( ctx->slot, fd_ulong_if( ctx->highwater_leader_slot==ULONG_MAX, 0UL, ctx->highwater_leader_slot ) );
1266 0 : return fd_multi_epoch_leaders_get_next_slot( ctx->mleaders, min_leader_slot, &ctx->identity_key );
1267 0 : }
1268 :
1269 : extern int
1270 : fd_ext_admin_rpc_set_identity( uchar const * identity_keypair,
1271 : int require_tower,
1272 : int require_vote_history );
1273 :
1274 : static inline int FD_FN_SENSITIVE
1275 : maybe_change_identity( fd_pohh_tile_t * ctx,
1276 0 : int definitely_not_leader ) {
1277 : /* Even if we are unhalted, still ack the request. This is because
1278 : if keyswitch fails, then the set identity process will send an
1279 : unhalt request and it needs to be processed. */
1280 0 : if( FD_UNLIKELY( fd_keyswitch_state_query( ctx->keyswitch )==FD_KEYSWITCH_STATE_UNHALT_PENDING ) ) {
1281 0 : ctx->halted_switching_key = 0;
1282 0 : fd_keyswitch_state( ctx->keyswitch, FD_KEYSWITCH_STATE_COMPLETED );
1283 0 : return 1;
1284 0 : }
1285 :
1286 : /* Cannot change identity while in the middle of a leader slot, else
1287 : poh state machine would become corrupt. */
1288 :
1289 0 : int is_leader = !definitely_not_leader && ctx->next_leader_slot!=ULONG_MAX && ctx->slot>=ctx->next_leader_slot;
1290 0 : if( FD_UNLIKELY( is_leader ) ) return 0;
1291 :
1292 0 : if( FD_UNLIKELY( fd_keyswitch_state_query( ctx->keyswitch )==FD_KEYSWITCH_STATE_SWITCH_PENDING ) ) {
1293 0 : ulong param = fd_keyswitch_param_query( ctx->keyswitch );
1294 0 : int failed = fd_ext_admin_rpc_set_identity( ctx->keyswitch->bytes, (int)(param & 1UL), (int)((param>>1) & 1UL) );
1295 0 : fd_memzero_explicit( ctx->keyswitch->bytes, 32UL );
1296 0 : FD_COMPILER_MFENCE();
1297 0 : if( FD_UNLIKELY( failed==-1 ) ) {
1298 0 : fd_keyswitch_state( ctx->keyswitch, FD_KEYSWITCH_STATE_FAILED );
1299 0 : return 0;
1300 0 : }
1301 :
1302 0 : memcpy( ctx->identity_key.uc, ctx->keyswitch->bytes+32UL, 32UL );
1303 :
1304 : /* When we switch key, we might have ticked part way through a slot
1305 : that we are now leader in. This violates the contract of the
1306 : tile, that when we become leader, we have not ticked in that slot
1307 : at all. To see why this would be bad, consider the case where we
1308 : have ticked almost to the end, and there isn't enough space left
1309 : to reserve the minimum amount of microblocks needed by pack.
1310 :
1311 : To resolve this, we just reset PoH back to the reset slot, and
1312 : let it try to catch back up quickly. This is OK since the network
1313 : rarely skips. */
1314 0 : ctx->slot = ctx->reset_slot;
1315 0 : ctx->hashcnt = 0UL;
1316 0 : memcpy( ctx->hash, ctx->reset_hash, 32UL );
1317 :
1318 0 : ctx->halted_switching_key = 1;
1319 0 : ctx->keyswitch->result = ctx->shred_seq;
1320 0 : fd_keyswitch_state( ctx->keyswitch, FD_KEYSWITCH_STATE_COMPLETED );
1321 0 : }
1322 :
1323 0 : return 0;
1324 0 : }
1325 :
1326 : static CALLED_FROM_RUST void
1327 0 : no_longer_leader( fd_pohh_tile_t * ctx ) {
1328 : /* If we acquired a bank for the store tile and never produced its
1329 : block_complete entry, the store tile never gets the chance to
1330 : drop the refcount, so we drop it directly here. */
1331 0 : if( FD_UNLIKELY( ctx->store_leader_bank_slot!=ULONG_MAX ) ) {
1332 0 : fd_ext_bank_release( ctx->store_leader_bank );
1333 0 : ctx->store_leader_bank = NULL;
1334 0 : ctx->store_leader_bank_slot = ULONG_MAX;
1335 0 : }
1336 :
1337 0 : if( FD_UNLIKELY( ctx->current_leader_bank ) ) fd_ext_bank_release( ctx->current_leader_bank );
1338 : /* If we stop being leader in a slot, we can never become leader in
1339 : that slot again, and all in-flight microblocks for that slot
1340 : should be dropped. */
1341 0 : ctx->highwater_leader_slot = fd_ulong_max( fd_ulong_if( ctx->highwater_leader_slot==ULONG_MAX, 0UL, ctx->highwater_leader_slot ), ctx->slot );
1342 0 : ctx->current_leader_bank = NULL;
1343 0 : int identity_changed = maybe_change_identity( ctx, 1 );
1344 0 : ctx->next_leader_slot = next_leader_slot( ctx );
1345 0 : if( FD_UNLIKELY( identity_changed ) ) {
1346 0 : FD_LOG_INFO(( "fd_poh_identity_changed(next_leader_slot=%lu)", ctx->next_leader_slot ));
1347 0 : }
1348 :
1349 0 : FD_COMPILER_MFENCE();
1350 0 : fd_ext_poh_signal_leader_change( ctx->signal_leader_change );
1351 0 : FD_LOG_INFO(( "no_longer_leader(next_leader_slot=%lu)", ctx->next_leader_slot ));
1352 0 : }
1353 :
1354 : /* fd_ext_poh_reset is called by the Agave client when a slot on
1355 : the active fork has finished a block and we need to reset our PoH to
1356 : be ticking on top of the block it produced. */
1357 :
1358 : CALLED_FROM_RUST void
1359 : fd_ext_poh_reset( ulong completed_bank_slot, /* The slot that successfully produced a block */
1360 : uchar const * reset_blockhash, /* The hash of the last tick in the produced block */
1361 : ulong hashcnt_per_tick, /* The hashcnt per tick of the bank that completed */
1362 : ulong tick_duration_ns, /* The target tick duration of the bank that completed */
1363 : uchar const * parent_block_id, /* The block id of the parent block */
1364 : ulong const * features_activation, /* The activation slot of shred-tile features */
1365 0 : ulong const * shred_slot_limits /* The shred slot limits for the epoch */ ) {
1366 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
1367 :
1368 0 : ulong slot_before_reset = ctx->slot;
1369 0 : int leader_before_reset = ctx->slot>=ctx->next_leader_slot;
1370 0 : if( FD_UNLIKELY( leader_before_reset && ctx->current_leader_bank ) ) {
1371 : /* If we were in the middle of a leader slot that we notified pack
1372 : pack to start packing for we can never publish into that slot
1373 : again, mark all in-flight microblocks to be dropped. */
1374 0 : ctx->highwater_leader_slot = fd_ulong_max( fd_ulong_if( ctx->highwater_leader_slot==ULONG_MAX, 0UL, ctx->highwater_leader_slot ), 1UL+ctx->slot );
1375 0 : }
1376 :
1377 0 : ctx->leader_bank_start_ns = fd_log_wallclock(); /* safe to call from Rust */
1378 0 : if( FD_UNLIKELY( ctx->expect_sequential_leader_slot==(completed_bank_slot+1UL) ) ) {
1379 : /* If we are being reset onto a slot, it means some block was fully
1380 : processed, so we reset to build on top of it. Typically we want
1381 : to update the reset_slot_start_ns to the current time, because
1382 : the network will give the next leader 400ms to publish,
1383 : regardless of how long the prior leader took.
1384 :
1385 : But: if we were leader in the prior slot, and the block was our
1386 : own we can do better. We know that the next slot should start
1387 : exactly 400ms after the prior one started, so we can use that as
1388 : the reset slot start time instead. */
1389 0 : ctx->reset_slot_start_ns = ctx->reset_slot_start_ns + (long)((double)((completed_bank_slot+1UL)-ctx->reset_slot)*ctx->slot_duration_ns);
1390 0 : } else {
1391 0 : ctx->reset_slot_start_ns = ctx->leader_bank_start_ns;
1392 0 : }
1393 0 : ctx->expect_sequential_leader_slot = ULONG_MAX;
1394 :
1395 0 : memcpy( ctx->reset_hash, reset_blockhash, 32UL );
1396 0 : memcpy( ctx->hash, reset_blockhash, 32UL );
1397 0 : if( FD_LIKELY( parent_block_id!=NULL ) ) {
1398 0 : ctx->parent_slot = completed_bank_slot;
1399 0 : memcpy( ctx->parent_block_id, parent_block_id, 32UL );
1400 0 : }
1401 0 : ctx->slot = completed_bank_slot+1UL;
1402 0 : ctx->hashcnt = 0UL;
1403 0 : ctx->last_slot = ctx->slot;
1404 0 : ctx->last_hashcnt = 0UL;
1405 0 : ctx->reset_slot = ctx->slot;
1406 :
1407 0 : if( FD_UNLIKELY( ctx->tick_duration_ns!=tick_duration_ns ) ) {
1408 0 : FD_LOG_WARNING(( "tick duration changed from %lu to %lu", ctx->tick_duration_ns, tick_duration_ns ));
1409 :
1410 : /* Recompute derived information about the clock. */
1411 0 : ctx->tick_duration_ns = tick_duration_ns;
1412 0 : ctx->slot_duration_ns = (double)ctx->ticks_per_slot*(double)tick_duration_ns;
1413 0 : ctx->hashcnt_duration_ns = (double)tick_duration_ns/(double)hashcnt_per_tick;
1414 0 : }
1415 :
1416 0 : if( FD_UNLIKELY( ctx->hashcnt_per_tick!=hashcnt_per_tick ) ) {
1417 0 : FD_LOG_WARNING(( "hashes per tick changed from %lu to %lu", ctx->hashcnt_per_tick, hashcnt_per_tick ));
1418 :
1419 : /* Recompute derived information about the clock. */
1420 0 : ctx->hashcnt_duration_ns = (double)ctx->tick_duration_ns/(double)hashcnt_per_tick;
1421 0 : ctx->hashcnt_per_slot = ctx->ticks_per_slot*hashcnt_per_tick;
1422 0 : ctx->hashcnt_per_tick = hashcnt_per_tick;
1423 0 : }
1424 :
1425 0 : if( FD_UNLIKELY( ctx->hashcnt_per_tick==1UL ) ) {
1426 : /* Low power producer, maximum of one microblock per tick in the slot */
1427 0 : ctx->max_microblocks_per_slot = ctx->ticks_per_slot;
1428 0 : } else {
1429 : /* See the long comment in after_credit for this limit */
1430 0 : ctx->max_microblocks_per_slot = fd_ulong_min( MAX_MICROBLOCKS_PER_SLOT, ctx->ticks_per_slot*(ctx->hashcnt_per_tick-1UL) );
1431 0 : }
1432 :
1433 : /* When we reset, we need to allow PoH to tick freely again rather
1434 : than being constrained. If we are leader after the reset, this
1435 : is OK because we won't tick until we get a bank, and the lower
1436 : bound will be reset with the value from the bank. */
1437 0 : ctx->microblocks_lower_bound = ctx->max_microblocks_per_slot;
1438 :
1439 0 : if( FD_UNLIKELY( leader_before_reset ) ) {
1440 : /* No longer have a leader bank if we are reset. Replay stage will
1441 : call back again to give us a new one if we should become leader
1442 : for the reset slot.
1443 :
1444 : The order is important here, ctx->hashcnt must be updated before
1445 : calling no_longer_leader. */
1446 0 : no_longer_leader( ctx );
1447 0 : }
1448 0 : ctx->next_leader_slot = next_leader_slot( ctx );
1449 0 : FD_LOG_INFO(( "fd_ext_poh_reset(slot=%lu,next_leader_slot=%lu)", ctx->reset_slot, ctx->next_leader_slot ));
1450 :
1451 0 : if( FD_UNLIKELY( ctx->slot>=ctx->next_leader_slot ) ) {
1452 : /* We are leader after the reset... two cases: */
1453 0 : if( FD_LIKELY( ctx->slot==slot_before_reset ) ) {
1454 : /* 1. We are reset onto the same slot we are already leader on.
1455 : This is a common case when we have two leader slots in a
1456 : row, replay stage will reset us to our own slot. No need to
1457 : do anything here, we already sent a SLOT_START. */
1458 0 : FD_TEST( leader_before_reset );
1459 0 : } else {
1460 : /* 2. We are reset onto a different slot. If we were leader
1461 : before, we should first end that slot, then begin the new
1462 : one if we are newly leader now. */
1463 0 : if( FD_LIKELY( leader_before_reset ) ) publish_plugin_slot_end( ctx, slot_before_reset, ctx->cus_used );
1464 0 : else publish_plugin_slot_start( ctx, ctx->next_leader_slot, ctx->reset_slot );
1465 0 : }
1466 0 : } else {
1467 0 : if( FD_UNLIKELY( leader_before_reset ) ) publish_plugin_slot_end( ctx, slot_before_reset, ctx->cus_used );
1468 0 : }
1469 :
1470 : /* There is a subset of FD_SHRED_FEATURES_ACTIVATION_... slots that
1471 : the shred tile needs to be aware of, as well as shred limits that
1472 : can change at epoch boundaries. Since their computation
1473 : requires the bank, we are forced (so far) to receive them here
1474 : from the Rust side, before forwarding them to the shred tile as
1475 : POH_PKT_TYPE_SHRED_EPOCH_MSG. This is not elegant, and it should
1476 : be revised in the future (TODO), but it provides a "temporary"
1477 : working solution. */
1478 0 : if( FD_UNLIKELY( !fd_memeq( &ctx->shred_epoch_msg->features_activation, features_activation, sizeof(fd_shred_features_activation_t) ) ) ) {
1479 0 : fd_memcpy( &ctx->shred_epoch_msg->features_activation, features_activation, sizeof(fd_shred_features_activation_t) );
1480 0 : ctx->shred_epoch_msg_avail = 1UL;
1481 0 : }
1482 0 : if( FD_UNLIKELY( !fd_memeq( &ctx->shred_epoch_msg->slot_limits, shred_slot_limits, sizeof(fd_shred_slot_limits_t) ) ) ) {
1483 0 : fd_memcpy( &ctx->shred_epoch_msg->slot_limits, shred_slot_limits, sizeof(fd_shred_slot_limits_t) );
1484 0 : ctx->shred_epoch_msg_avail = 1UL;
1485 0 : }
1486 :
1487 0 : fd_ext_poh_write_unlock();
1488 0 : }
1489 :
1490 : /* Since it can't easily return an Option<Pubkey>, return 1 for Some and
1491 : 0 for None. */
1492 : CALLED_FROM_RUST int
1493 : fd_ext_poh_get_leader_after_n_slots( ulong n,
1494 0 : uchar out_pubkey[ static 32 ] ) {
1495 0 : fd_pohh_tile_t * ctx = fd_ext_poh_write_lock();
1496 0 : ulong slot = ctx->slot + n;
1497 0 : fd_pubkey_t const * leader = fd_multi_epoch_leaders_get_leader_for_slot( ctx->mleaders, slot );
1498 :
1499 0 : int copied = 0;
1500 0 : if( FD_LIKELY( leader ) ) {
1501 0 : memcpy( out_pubkey, leader, 32UL );
1502 0 : copied = 1;
1503 0 : }
1504 0 : fd_ext_poh_write_unlock();
1505 0 : return copied;
1506 0 : }
1507 :
1508 : FD_FN_CONST static inline ulong
1509 0 : scratch_align( void ) {
1510 0 : return 128UL;
1511 0 : }
1512 :
1513 : FD_FN_PURE static inline ulong
1514 0 : scratch_footprint( fd_topo_tile_t const * tile ) {
1515 0 : (void)tile;
1516 0 : ulong l = FD_LAYOUT_INIT;
1517 0 : l = FD_LAYOUT_APPEND( l, alignof( fd_pohh_tile_t ), sizeof( fd_pohh_tile_t ) );
1518 0 : l = FD_LAYOUT_APPEND( l, FD_SHA256_ALIGN, FD_SHA256_FOOTPRINT );
1519 0 : return FD_LAYOUT_FINI( l, scratch_align() );
1520 0 : }
1521 :
1522 : static void
1523 : publish_tick( fd_pohh_tile_t * ctx,
1524 : fd_stem_context_t * stem,
1525 : uchar hash[ static 32 ],
1526 0 : int is_skipped ) {
1527 0 : ulong hashcnt = ctx->hashcnt_per_tick*(1UL+(ctx->last_hashcnt/ctx->hashcnt_per_tick));
1528 :
1529 0 : uchar * dst = (uchar *)fd_chunk_to_laddr( ctx->shred_out->mem, ctx->shred_out->chunk );
1530 :
1531 0 : FD_TEST( ctx->last_slot>=ctx->reset_slot );
1532 0 : fd_entry_batch_meta_t * meta = (fd_entry_batch_meta_t *)dst;
1533 0 : if( FD_UNLIKELY( is_skipped ) ) {
1534 : /* We are publishing ticks for a skipped slot, the reference tick
1535 : and block complete flags should always be zero. */
1536 0 : meta->reference_tick = 0UL;
1537 0 : meta->block_complete = 0;
1538 0 : } else {
1539 0 : meta->reference_tick = hashcnt/ctx->hashcnt_per_tick;
1540 0 : meta->block_complete = hashcnt==ctx->hashcnt_per_slot;
1541 0 : }
1542 :
1543 0 : ulong slot = fd_ulong_if( meta->block_complete, ctx->slot-1UL, ctx->slot );
1544 0 : meta->parent_offset = 1UL+slot-ctx->reset_slot;
1545 :
1546 : /* From poh_reset we received the block_id for ctx->parent_slot.
1547 : Now we're telling shred tile to build on parent: (slot-meta->parent_offset).
1548 : The block_id that we're passing is valid iff the two are the same,
1549 : i.e. ctx->parent_slot == (slot-meta->parent_offset). */
1550 0 : meta->parent_block_id_valid = ctx->parent_slot == (slot-meta->parent_offset);
1551 0 : if( FD_LIKELY( meta->parent_block_id_valid ) ) {
1552 0 : fd_memcpy( meta->parent_block_id, ctx->parent_block_id, 32UL );
1553 0 : }
1554 :
1555 0 : FD_TEST( hashcnt>ctx->last_hashcnt );
1556 0 : ulong hash_delta = hashcnt-ctx->last_hashcnt;
1557 :
1558 0 : dst += sizeof(fd_entry_batch_meta_t);
1559 0 : fd_entry_batch_header_t * tick = (fd_entry_batch_header_t *)dst;
1560 0 : tick->hashcnt_delta = hash_delta;
1561 0 : fd_memcpy( tick->hash, hash, 32UL );
1562 0 : tick->txn_cnt = 0UL;
1563 :
1564 0 : ulong tspub = (ulong)fd_frag_meta_ts_comp( fd_tickcount() );
1565 0 : ulong sz = sizeof(fd_entry_batch_meta_t)+sizeof(fd_entry_batch_header_t);
1566 0 : ulong sig = fd_disco_poh_sig( slot, POH_PKT_TYPE_MICROBLOCK, 0UL );
1567 0 : fd_stem_publish( stem, ctx->shred_out->idx, sig, ctx->shred_out->chunk, sz, 0UL, 0UL, tspub );
1568 0 : ctx->shred_seq = stem->seqs[ ctx->shred_out->idx ];
1569 0 : ctx->shred_out->chunk = fd_dcache_compact_next( ctx->shred_out->chunk, sz, ctx->shred_out->chunk0, ctx->shred_out->wmark );
1570 :
1571 0 : if( FD_UNLIKELY( hashcnt==ctx->hashcnt_per_slot ) ) {
1572 0 : ctx->last_slot++;
1573 0 : ctx->last_hashcnt = 0UL;
1574 0 : } else {
1575 0 : ctx->last_hashcnt = hashcnt;
1576 0 : }
1577 :
1578 : /* The store tile will release the bank refcount when an FEC set with
1579 : SLOT_COMPLETE arrives, so we drop our local tracking and suppress
1580 : the END type frag in no_longer_leader. */
1581 0 : if( FD_UNLIKELY( meta->block_complete && ctx->store_leader_bank_slot!=ULONG_MAX ) ) {
1582 0 : FD_TEST( ctx->store_leader_bank_slot==slot );
1583 0 : ctx->store_leader_bank = NULL;
1584 0 : ctx->store_leader_bank_slot = ULONG_MAX;
1585 0 : }
1586 0 : }
1587 :
1588 : static inline void
1589 : publish_shred_epoch_msg( fd_pohh_tile_t * ctx,
1590 0 : fd_stem_context_t * stem ) {
1591 0 : fd_shred_epoch_msg_t * emsg = (fd_shred_epoch_msg_t *)fd_chunk_to_laddr( ctx->shred_out->mem, ctx->shred_out->chunk );
1592 0 : *emsg = *ctx->shred_epoch_msg;
1593 :
1594 0 : ulong tspub = (ulong)fd_frag_meta_ts_comp( fd_tickcount() );
1595 0 : ulong sz = sizeof(fd_shred_epoch_msg_t);
1596 0 : ulong sig = fd_disco_poh_sig( ctx->slot, POH_PKT_TYPE_SHRED_EPOCH_MSG, 0UL );
1597 0 : fd_stem_publish( stem, ctx->shred_out->idx, sig, ctx->shred_out->chunk, sz, 0UL, 0UL, tspub );
1598 0 : ctx->shred_seq = stem->seqs[ ctx->shred_out->idx ];
1599 0 : ctx->shred_out->chunk = fd_dcache_compact_next( ctx->shred_out->chunk, sz, ctx->shred_out->chunk0, ctx->shred_out->wmark );
1600 0 : }
1601 :
1602 : static inline void
1603 : after_credit( fd_pohh_tile_t * ctx,
1604 : fd_stem_context_t * stem,
1605 : int * opt_poll_in,
1606 0 : int * charge_busy ) {
1607 0 : ctx->stem = stem;
1608 :
1609 0 : FD_COMPILER_MFENCE();
1610 0 : if( FD_UNLIKELY( fd_poh_waiting_lock ) ) {
1611 0 : FD_VOLATILE( fd_poh_returned_lock ) = 1UL;
1612 0 : FD_COMPILER_MFENCE();
1613 0 : for(;;) {
1614 0 : if( FD_UNLIKELY( !FD_VOLATILE_CONST( fd_poh_returned_lock ) ) ) break;
1615 0 : FD_SPIN_PAUSE();
1616 0 : }
1617 0 : FD_COMPILER_MFENCE();
1618 0 : FD_VOLATILE( fd_poh_waiting_lock ) = 0UL;
1619 0 : *opt_poll_in = 0;
1620 0 : *charge_busy = 1;
1621 0 : return;
1622 0 : }
1623 0 : FD_COMPILER_MFENCE();
1624 :
1625 0 : if( FD_UNLIKELY( ctx->shred_epoch_msg_avail ) ) {
1626 : /* If we received a fresh feature activations or shred slot limits,
1627 : we need to forward this to the shred tile. In principle, this
1628 : happens at most once per slot. */
1629 0 : publish_shred_epoch_msg( ctx, stem );
1630 0 : ctx->shred_epoch_msg_avail = 0UL;
1631 0 : }
1632 :
1633 0 : int is_leader = ctx->next_leader_slot!=ULONG_MAX && ctx->slot>=ctx->next_leader_slot;
1634 0 : if( FD_UNLIKELY( is_leader && !ctx->current_leader_bank ) ) {
1635 : /* If we are the leader, but we didn't yet learn what the leader
1636 : bank object is from the replay stage, do not do any hashing.
1637 :
1638 : This is not ideal, but greatly simplifies the control flow. */
1639 0 : return;
1640 0 : }
1641 :
1642 : /* If we have skipped ticks pending because we skipped some slots to
1643 : become leader, register them now one at a time. */
1644 0 : if( FD_UNLIKELY( is_leader && ctx->last_slot<ctx->slot ) ) {
1645 0 : ulong publish_hashcnt = ctx->last_hashcnt+ctx->hashcnt_per_tick;
1646 0 : ulong tick_idx = (ctx->last_slot*ctx->ticks_per_slot+publish_hashcnt/ctx->hashcnt_per_tick)%MAX_SKIPPED_TICKS;
1647 :
1648 0 : fd_ext_poh_register_tick( ctx->current_leader_bank, ctx->skipped_tick_hashes[ tick_idx ] );
1649 0 : publish_tick( ctx, stem, ctx->skipped_tick_hashes[ tick_idx ], 1 );
1650 :
1651 : /* If we are catching up now and publishing a bunch of skipped
1652 : ticks, we do not want to process any incoming microblocks until
1653 : all the skipped ticks have been published out; otherwise we would
1654 : intersperse skipped tick messages with microblocks. */
1655 0 : *opt_poll_in = 0;
1656 0 : *charge_busy = 1;
1657 0 : return;
1658 0 : }
1659 :
1660 0 : int low_power_mode = ctx->hashcnt_per_tick==1UL;
1661 :
1662 : /* If we are the leader, always leave enough capacity in the slot so
1663 : that we can mixin any potential microblocks still coming from the
1664 : pack tile for this slot. */
1665 0 : ulong max_remaining_microblocks = ctx->max_microblocks_per_slot - ctx->microblocks_lower_bound;
1666 :
1667 : /* With hashcnt_per_tick hashes per tick, we actually get
1668 : hashcnt_per_tick-1 chances to mixin a microblock. For each tick
1669 : span that we need to reserve, we also need to reserve the hashcnt
1670 : for the tick, hence the +
1671 : max_remaining_microblocks/(hashcnt_per_tick-1) rounded up.
1672 :
1673 : However, if hashcnt_per_tick is 1 because we're in low power mode,
1674 : this should probably just be max_remaining_microblocks. */
1675 0 : ulong max_remaining_ticks_or_microblocks = max_remaining_microblocks;
1676 0 : if( FD_LIKELY( !low_power_mode ) ) max_remaining_ticks_or_microblocks += (max_remaining_microblocks+ctx->hashcnt_per_tick-2UL)/(ctx->hashcnt_per_tick-1UL);
1677 :
1678 0 : ulong restricted_hashcnt = fd_ulong_if( ctx->hashcnt_per_slot>=max_remaining_ticks_or_microblocks, ctx->hashcnt_per_slot-max_remaining_ticks_or_microblocks, 0UL );
1679 :
1680 0 : ulong min_hashcnt = ctx->hashcnt;
1681 :
1682 0 : if( FD_LIKELY( !low_power_mode ) ) {
1683 : /* Recall that there are two kinds of events that will get published
1684 : to the shredder,
1685 :
1686 : (a) Ticks. These occur every 62,500 (hashcnt_per_tick) hashcnts,
1687 : and there will be 64 (ticks_per_slot) of them in each slot.
1688 :
1689 : Ticks must not have any transactions mixed into the hash.
1690 : This is not strictly needed in theory, but is required by the
1691 : current consensus protocol. They get published here in
1692 : after_credit.
1693 :
1694 : (b) Microblocks. These can occur at any other hashcnt, as long
1695 : as it is not a tick. Microblocks cannot be empty, and must
1696 : have at least one transactions mixed in. These get
1697 : published in after_frag.
1698 :
1699 : If hashcnt_per_tick is 1, then we are in low power mode and the
1700 : following does not apply, since we can mix in transactions at any
1701 : time.
1702 :
1703 : In the normal, non-low-power mode, though, we have to be careful
1704 : to make sure that we do not publish microblocks on tick
1705 : boundaries. To do that, we need to obey two rules:
1706 : (i) after_credit must not leave hashcnt one before a tick
1707 : boundary
1708 : (ii) if after_credit begins one before a tick boundary, it must
1709 : advance hashcnt and publish the tick
1710 :
1711 : There's some interplay between min_hashcnt and restricted_hashcnt
1712 : here, and we need to show that there's always a value of
1713 : target_hashcnt we can pick such that
1714 : min_hashcnt <= target_hashcnt <= restricted_hashcnt.
1715 : We'll prove this by induction for current_slot==0 and
1716 : is_leader==true, since all other slots should be the same.
1717 :
1718 : Let m_j and r_j be the min_hashcnt and restricted_hashcnt
1719 : (respectively) for the jth call to after_credit in a slot. We
1720 : want to show that for all values of j, it's possible to pick a
1721 : value h_j, the value of target_hashcnt for the jth call to
1722 : after_credit (which is also the value of hashcnt after
1723 : after_credit has completed) such that m_j<=h_j<=r_j.
1724 :
1725 : Additionally, let T be hashcnt_per_tick and N be ticks_per_slot.
1726 :
1727 : Starting with the base case, j==0. m_j=0, and
1728 : r_0 = N*T - max_microblocks_per_slot
1729 : - ceil(max_microblocks_per_slot/(T-1)).
1730 :
1731 : This is monotonic decreasing in max_microblocks_per_slot, so it
1732 : achieves its minimum when max_microblocks_per_slot is its
1733 : maximum.
1734 : r_0 >= N*T - N*(T-1) - ceil( (N*(T-1))/(T-1))
1735 : = N*T - N*(T-1)-N = 0.
1736 : Thus, m_0 <= r_0, as desired.
1737 :
1738 :
1739 :
1740 : Then, for the inductive step, assume there exists h_j such that
1741 : m_j<=h_j<=r_j, and we want to show that there exists h_{j+1},
1742 : which is the same as showing m_{j+1}<=r_{j+1}.
1743 :
1744 : Let a_j be 1 if we had a microblock immediately following the jth
1745 : call to after_credit, and 0 otherwise. Then hashcnt at the start
1746 : of the (j+1)th call to after_frag is h_j+a_j.
1747 : Also, set b_{j+1}=1 if we are in the case covered by rule (ii)
1748 : above during the (j+1)th call to after_credit, i.e. if
1749 : (h_j+a_j)%T==T-1. Thus, m_{j+1} = h_j + a_j + b_{j+1}.
1750 :
1751 : If we received an additional microblock, then
1752 : max_remaining_microblocks goes down by 1, and
1753 : max_remaining_ticks_or_microblocks goes down by either 1 or 2,
1754 : which means restricted_hashcnt goes up by either 1 or 2. In
1755 : particular, it goes up by 2 if the new value of
1756 : max_remaining_microblocks (at the start of the (j+1)th call to
1757 : after_credit) is congruent to 0 mod T-1. Let b'_{j+1} be 1 if
1758 : this condition is met and 0 otherwise. If we receive a
1759 : done_packing message, restricted_hashcnt can go up by more, but
1760 : we can ignore that case, since it is less restrictive.
1761 : Thus, r_{j+1}=r_j+a_j+b'_{j+1}.
1762 :
1763 : If h_j < r_j (strictly less), then h_j+a_j < r_j+a_j. And thus,
1764 : since b_{j+1}<=b'_{j+1}+1, just by virtue of them both being
1765 : binary,
1766 : h_j + a_j + b_{j+1} < r_j + a_j + b'_{j+1} + 1,
1767 : which is the same (for integers) as
1768 : h_j + a_j + b_{j+1} <= r_j + a_j + b'_{j+1},
1769 : m_{j+1} <= r_{j+1}
1770 :
1771 : On the other hand, if h_j==r_j, this is easy unless b_{j+1}==1,
1772 : which can also only happen if a_j==1. Then (h_j+a_j)%T==T-1,
1773 : which means there's an integer k such that
1774 :
1775 : h_j+a_j==(ticks_per_slot-k)*T-1
1776 : h_j ==ticks_per_slot*T - k*(T-1)-1 - k-1
1777 : ==ticks_per_slot*T - (k*(T-1)+1) - ceil( (k*(T-1)+1)/(T-1) )
1778 :
1779 : Since h_j==r_j in this case, and
1780 : r_j==(ticks_per_slot*T) - max_remaining_microblocks_j - ceil(max_remaining_microblocks_j/(T-1)),
1781 : we can see that the value of max_remaining_microblocks at the
1782 : start of the jth call to after_credit is k*(T-1)+1. Again, since
1783 : a_j==1, then the value of max_remaining_microblocks at the start
1784 : of the j+1th call to after_credit decreases by 1 to k*(T-1),
1785 : which means b'_{j+1}=1.
1786 :
1787 : Thus, h_j + a_j + b_{j+1} == r_j + a_j + b'_{j+1}, so, in
1788 : particular, h_{j+1}<=r_{j+1} as desired. */
1789 0 : min_hashcnt += (ulong)(min_hashcnt%ctx->hashcnt_per_tick == (ctx->hashcnt_per_tick-1UL)); /* add b_{j+1}, enforcing rule (ii) */
1790 0 : }
1791 : /* Now figure out how many hashes are needed to "catch up" the hash
1792 : count to the current system clock, and clamp it to the allowed
1793 : range. */
1794 0 : long now = fd_log_wallclock();
1795 0 : ulong target_hashcnt;
1796 0 : if( FD_LIKELY( !is_leader ) ) {
1797 0 : target_hashcnt = (ulong)((double)(now - ctx->reset_slot_start_ns) / ctx->hashcnt_duration_ns) - (ctx->slot-ctx->reset_slot)*ctx->hashcnt_per_slot;
1798 0 : } else {
1799 : /* We might have gotten very behind on hashes, but if we are leader
1800 : we want to catch up gradually over the remainder of our leader
1801 : slot, not all at once right now. This helps keep the tile from
1802 : being oversubscribed and taking a long time to process incoming
1803 : microblocks. */
1804 0 : long expected_slot_start_ns = ctx->reset_slot_start_ns + (long)((double)(ctx->slot-ctx->reset_slot)*ctx->slot_duration_ns);
1805 0 : double actual_slot_duration_ns = ctx->slot_duration_ns<(double)(ctx->leader_bank_start_ns - expected_slot_start_ns) ? 0.0 : ctx->slot_duration_ns - (double)(ctx->leader_bank_start_ns - expected_slot_start_ns);
1806 0 : double actual_hashcnt_duration_ns = actual_slot_duration_ns / (double)ctx->hashcnt_per_slot;
1807 0 : target_hashcnt = actual_hashcnt_duration_ns==0.0 ? restricted_hashcnt : (ulong)((double)(now - ctx->leader_bank_start_ns) / actual_hashcnt_duration_ns);
1808 0 : }
1809 : /* Clamp to [min_hashcnt, restricted_hashcnt] as above */
1810 0 : target_hashcnt = fd_ulong_max( fd_ulong_min( target_hashcnt, restricted_hashcnt ), min_hashcnt );
1811 :
1812 : /* The above proof showed that it was always possible to pick a value
1813 : of target_hashcnt, but we still have a lot of freedom in how to
1814 : pick it. It simplifies the code a lot if we don't keep going after
1815 : a tick in this function. In particular, we want to publish at most
1816 : 1 tick in this call, since otherwise we could consume infinite
1817 : credits to publish here. The credits are set so that we should
1818 : only ever publish one tick during this loop. Also, all the extra
1819 : stuff (leader transitions, publishing ticks, etc.) we have to do
1820 : happens at tick boundaries, so this lets us consolidate all those
1821 : cases.
1822 :
1823 : Mathematically, since the current value of hashcnt is h_j+a_j, the
1824 : next tick (advancing a full tick if we're currently at a tick) is
1825 : t_{j+1} = T*(floor( (h_j+a_j)/T )+1). We need to show that if we set
1826 : h'_{j+1} = min( h_{j+1}, t_{j+1} ), it is still valid.
1827 :
1828 : First, h'_{j+1} <= h_{j+1} <= r_{j+1}, so we're okay in that
1829 : direction.
1830 :
1831 : Next, observe that t_{j+1}>=h_j + a_j + 1, and recall that b_{j+1}
1832 : is 0 or 1. So then,
1833 : t_{j+1} >= h_j+a_j+b_{j+1} = m_{j+1}.
1834 :
1835 : We know h_{j+1) >= m_{j+1} from before, so then h'_{j+1} >=
1836 : m_{j+1}, as desired. */
1837 :
1838 0 : ulong next_tick_hashcnt = ctx->hashcnt_per_tick * (1UL+(ctx->hashcnt/ctx->hashcnt_per_tick));
1839 0 : target_hashcnt = fd_ulong_min( target_hashcnt, next_tick_hashcnt );
1840 :
1841 : /* We still need to enforce rule (i). We know that min_hashcnt%T !=
1842 : T-1 because of rule (ii). That means that if target_hashcnt%T ==
1843 : T-1 at this point, target_hashcnt > min_hashcnt (notice the
1844 : strict), so target_hashcnt-1 >= min_hashcnt and is thus still a
1845 : valid choice for target_hashcnt. */
1846 0 : target_hashcnt -= (ulong)( (!low_power_mode) & ((target_hashcnt%ctx->hashcnt_per_tick)==(ctx->hashcnt_per_tick-1UL)) );
1847 :
1848 0 : FD_TEST( target_hashcnt >= ctx->hashcnt );
1849 0 : FD_TEST( target_hashcnt >= min_hashcnt );
1850 0 : FD_TEST( target_hashcnt <= restricted_hashcnt );
1851 :
1852 0 : if( FD_UNLIKELY( ctx->hashcnt==target_hashcnt ) ) return; /* Nothing to do, don't publish a tick twice */
1853 :
1854 0 : *charge_busy = 1;
1855 :
1856 0 : if( FD_LIKELY( ctx->hashcnt<target_hashcnt ) ) {
1857 0 : fd_sha256_hash_32_repeated( ctx->hash, ctx->hash, target_hashcnt-ctx->hashcnt );
1858 0 : ctx->hashcnt = target_hashcnt;
1859 0 : }
1860 :
1861 0 : if( FD_UNLIKELY( ctx->hashcnt==ctx->hashcnt_per_slot ) ) {
1862 0 : ctx->slot++;
1863 0 : ctx->hashcnt = 0UL;
1864 0 : }
1865 :
1866 0 : if( FD_UNLIKELY( !is_leader && !(ctx->hashcnt%ctx->hashcnt_per_tick ) ) ) {
1867 : /* We finished a tick while not leader... save the current hash so
1868 : it can be played back into the bank when we become the leader. */
1869 0 : ulong tick_idx = (ctx->slot*ctx->ticks_per_slot+ctx->hashcnt/ctx->hashcnt_per_tick)%MAX_SKIPPED_TICKS;
1870 0 : fd_memcpy( ctx->skipped_tick_hashes[ tick_idx ], ctx->hash, 32UL );
1871 :
1872 0 : ulong initial_tick_idx = (ctx->last_slot*ctx->ticks_per_slot+ctx->last_hashcnt/ctx->hashcnt_per_tick)%MAX_SKIPPED_TICKS;
1873 0 : if( FD_UNLIKELY( tick_idx==initial_tick_idx ) ) FD_LOG_ERR(( "Too many skipped ticks from slot %lu to slot %lu, chain must halt", ctx->last_slot, ctx->slot ));
1874 0 : }
1875 :
1876 0 : if( FD_UNLIKELY( is_leader && !(ctx->hashcnt%ctx->hashcnt_per_tick) ) ) {
1877 : /* We ticked while leader... tell the leader bank. */
1878 0 : fd_ext_poh_register_tick( ctx->current_leader_bank, ctx->hash );
1879 :
1880 : /* And send an empty microblock (a tick) to the shred tile. */
1881 0 : publish_tick( ctx, stem, ctx->hash, 0 );
1882 0 : }
1883 :
1884 0 : if( FD_UNLIKELY( !is_leader && ctx->slot>=ctx->next_leader_slot ) ) {
1885 : /* We ticked while not leader and are now leader... transition
1886 : the state machine. */
1887 0 : publish_plugin_slot_start( ctx, ctx->next_leader_slot, ctx->reset_slot );
1888 0 : FD_LOG_INFO(( "fd_poh_ticked_into_leader(slot=%lu, reset_slot=%lu)", ctx->next_leader_slot, ctx->reset_slot ));
1889 0 : }
1890 :
1891 0 : if( FD_UNLIKELY( is_leader && ctx->slot>ctx->next_leader_slot ) ) {
1892 : /* We ticked while leader and are no longer leader... transition
1893 : the state machine. */
1894 0 : FD_TEST( !max_remaining_microblocks );
1895 0 : publish_plugin_slot_end( ctx, ctx->next_leader_slot, ctx->cus_used );
1896 0 : FD_LOG_INFO(( "fd_poh_ticked_outof_leader(slot=%lu)", ctx->next_leader_slot ));
1897 :
1898 0 : no_longer_leader( ctx );
1899 0 : ctx->expect_sequential_leader_slot = ctx->slot;
1900 :
1901 0 : double tick_per_ns = fd_tempo_tick_per_ns( NULL );
1902 0 : fd_histf_sample( ctx->slot_done_delay, (ulong)((double)(fd_log_wallclock()-ctx->reset_slot_start_ns)*tick_per_ns) );
1903 0 : ctx->next_leader_slot = next_leader_slot( ctx );
1904 :
1905 0 : if( FD_UNLIKELY( ctx->slot>=ctx->next_leader_slot ) ) {
1906 : /* We finished a leader slot, and are immediately leader for the
1907 : following slot... transition. */
1908 0 : publish_plugin_slot_start( ctx, ctx->next_leader_slot, ctx->next_leader_slot-1UL );
1909 0 : FD_LOG_INFO(( "fd_poh_ticked_into_leader(slot=%lu, reset_slot=%lu)", ctx->next_leader_slot, ctx->next_leader_slot-1UL ));
1910 0 : }
1911 0 : }
1912 0 : }
1913 :
1914 : static inline void
1915 0 : during_housekeeping( fd_pohh_tile_t * ctx ) {
1916 0 : if( FD_UNLIKELY( maybe_change_identity( ctx, 0 ) ) ) {
1917 0 : ctx->next_leader_slot = next_leader_slot( ctx );
1918 0 : FD_LOG_INFO(( "fd_poh_identity_changed(next_leader_slot=%lu)", ctx->next_leader_slot ));
1919 :
1920 : /* Signal replay to check if we are leader again, in-case it's stuck
1921 : because everything already replayed. */
1922 0 : FD_COMPILER_MFENCE();
1923 0 : fd_ext_poh_signal_leader_change( ctx->signal_leader_change );
1924 0 : }
1925 0 : }
1926 :
1927 : static inline void
1928 0 : metrics_write( fd_pohh_tile_t * ctx ) {
1929 0 : FD_MHIST_COPY( POHH, BEGIN_LEADER_DELAY_SECONDS, ctx->begin_leader_delay );
1930 0 : FD_MHIST_COPY( POHH, FIRST_MICROBLOCK_DELAY_SECONDS, ctx->first_microblock_delay );
1931 0 : FD_MHIST_COPY( POHH, SLOT_DONE_DELAY_SECONDS, ctx->slot_done_delay );
1932 0 : FD_MHIST_COPY( POHH, BUNDLE_INITIALIZE_DELAY_SECONDS, ctx->bundle_init_delay );
1933 0 : }
1934 :
1935 : static int
1936 : before_frag( fd_pohh_tile_t * ctx,
1937 : ulong in_idx,
1938 : ulong seq,
1939 0 : ulong sig ) {
1940 0 : (void)seq;
1941 :
1942 0 : if( FD_LIKELY( ctx->in_kind[ in_idx ]!=IN_KIND_BANK && ctx->in_kind[ in_idx ]!=IN_KIND_PACK ) ) return 0;
1943 :
1944 0 : if( FD_UNLIKELY( sig==FD_PACK_MSG_DONE_DRAINING ) ) {
1945 : /* Banks are drained, release pack's ownership of the current bank */
1946 0 : if( FD_UNLIKELY( ctx->pack_leader_bank ) ) fd_ext_bank_release( ctx->pack_leader_bank );
1947 0 : ctx->pack_leader_bank = NULL;
1948 0 : return 1; /* discard */
1949 0 : }
1950 :
1951 : /* Firedancer publishes dynamic microblock bound updates over the
1952 : pack_poh link. Frankendancer does not use them. */
1953 0 : if( FD_UNLIKELY( sig==FD_PACK_MSG_REDUCE_MB_BOUND ) ) return 1; /* discard */
1954 :
1955 0 : uint pack_idx = (uint)fd_disco_execle_sig_pack_idx( sig );
1956 0 : FD_TEST( ((int)(pack_idx-ctx->expect_pack_idx))>=0L );
1957 0 : if( FD_UNLIKELY( pack_idx!=ctx->expect_pack_idx ) ) return -1;
1958 0 : ctx->expect_pack_idx++;
1959 :
1960 0 : return 0;
1961 0 : }
1962 :
1963 : static inline void
1964 : during_frag( fd_pohh_tile_t * ctx,
1965 : ulong in_idx,
1966 : ulong seq FD_PARAM_UNUSED,
1967 : ulong sig,
1968 : ulong chunk,
1969 : ulong sz,
1970 0 : ulong ctl FD_PARAM_UNUSED ) {
1971 0 : ctx->skip_frag = 0;
1972 :
1973 0 : if( FD_UNLIKELY( ctx->in_kind[ in_idx ]==IN_KIND_EPOCH ) ) {
1974 0 : if( FD_UNLIKELY( chunk<ctx->in[ in_idx ].chunk0 || chunk>ctx->in[ in_idx ].wmark ) )
1975 0 : FD_LOG_ERR(( "chunk %lu %lu corrupt, not in range [%lu,%lu]", chunk, sz,
1976 0 : ctx->in[ in_idx ].chunk0, ctx->in[ in_idx ].wmark ));
1977 :
1978 0 : uchar const * dcache_entry = fd_chunk_to_laddr_const( ctx->in[ in_idx ].mem, chunk );
1979 0 : fd_multi_epoch_leaders_stake_msg_init( ctx->mleaders, fd_type_pun_const( dcache_entry ) );
1980 0 : return;
1981 0 : }
1982 :
1983 0 : ulong slot;
1984 0 : switch( ctx->in_kind[ in_idx ] ) {
1985 0 : case IN_KIND_BANK:
1986 0 : case IN_KIND_PACK: {
1987 0 : slot = fd_disco_execle_sig_slot( sig );
1988 0 : break;
1989 0 : }
1990 0 : default:
1991 0 : FD_LOG_ERR(( "unexpected in_kind %d", ctx->in_kind[ in_idx ] ));
1992 0 : }
1993 :
1994 : /* The following sequence is possible...
1995 :
1996 : 1. We become leader in slot 10
1997 : 2. While leader, we switch to a fork that is on slot 8, where
1998 : we are leader
1999 : 3. We get the in-flight microblocks for slot 10
2000 :
2001 : These in-flight microblocks need to be dropped, so we check
2002 : against the high water mark (highwater_leader_slot) rather than
2003 : the current hashcnt here when determining what to drop.
2004 :
2005 : We know if the slot is lower than the high water mark it's from a stale
2006 : leader slot, because we will not become leader for the same slot twice
2007 : even if we are reset back in time (to prevent duplicate blocks). */
2008 0 : int is_frag_for_prior_leader_slot = slot<ctx->highwater_leader_slot;
2009 :
2010 0 : if( FD_UNLIKELY( ctx->in_kind[ in_idx ]==IN_KIND_PACK ) ) {
2011 : /* We now know the real amount of microblocks published, so set an
2012 : exact bound for once we receive them. */
2013 0 : ctx->skip_frag = 1;
2014 0 : if( FD_UNLIKELY( is_frag_for_prior_leader_slot ) ) return;
2015 0 : fd_done_packing_t const * done_packing = fd_chunk_to_laddr( ctx->in[ in_idx ].mem, chunk );
2016 :
2017 0 : FD_TEST( ctx->microblocks_lower_bound<=ctx->max_microblocks_per_slot );
2018 0 : FD_TEST( done_packing->microblocks_in_slot<=ctx->max_microblocks_per_slot-1UL );
2019 0 : FD_LOG_INFO(( "done_packing(slot=%lu,seen_microblocks=%lu,microblocks_in_slot=%lu)",
2020 0 : ctx->slot,
2021 0 : ctx->microblocks_lower_bound,
2022 0 : done_packing->microblocks_in_slot ));
2023 :
2024 0 : ctx->microblocks_lower_bound += 1UL /* done_packing as a phantom "microblock"*/
2025 0 : + (ctx->max_microblocks_per_slot-1UL) /* the canonical microblock limit */
2026 0 : - done_packing->microblocks_in_slot /* the actual microblock count */;
2027 0 : return;
2028 0 : } else {
2029 0 : if( FD_UNLIKELY( chunk<ctx->in[ in_idx ].chunk0 || chunk>ctx->in[ in_idx ].wmark || sz>USHORT_MAX ) )
2030 0 : FD_LOG_ERR(( "chunk %lu %lu corrupt, not in range [%lu,%lu]", chunk, sz, ctx->in[ in_idx ].chunk0, ctx->in[ in_idx ].wmark ));
2031 :
2032 0 : uchar * src = (uchar *)fd_chunk_to_laddr( ctx->in[ in_idx ].mem, chunk );
2033 :
2034 0 : fd_memcpy( ctx->_txns, src, sz-sizeof(fd_microblock_trailer_t) );
2035 0 : fd_memcpy( ctx->_microblock_trailer, src+sz-sizeof(fd_microblock_trailer_t), sizeof(fd_microblock_trailer_t) );
2036 :
2037 0 : ctx->skip_frag = is_frag_for_prior_leader_slot;
2038 0 : }
2039 0 : }
2040 :
2041 : static void
2042 : publish_microblock( fd_pohh_tile_t * ctx,
2043 : fd_stem_context_t * stem,
2044 : ulong slot,
2045 : ulong hashcnt_delta,
2046 0 : ulong txn_cnt ) {
2047 0 : uchar * dst = (uchar *)fd_chunk_to_laddr( ctx->shred_out->mem, ctx->shred_out->chunk );
2048 0 : FD_TEST( slot>=ctx->reset_slot );
2049 0 : fd_entry_batch_meta_t * meta = (fd_entry_batch_meta_t *)dst;
2050 0 : meta->parent_offset = 1UL+slot-ctx->reset_slot;
2051 0 : meta->reference_tick = (ctx->hashcnt/ctx->hashcnt_per_tick) % ctx->ticks_per_slot;
2052 0 : meta->block_complete = !ctx->hashcnt;
2053 :
2054 : /* Refer to publish_tick() for details on meta->parent_block_id_valid. */
2055 0 : meta->parent_block_id_valid = ctx->parent_slot == (slot-meta->parent_offset);
2056 0 : if( FD_LIKELY( meta->parent_block_id_valid ) ) {
2057 0 : fd_memcpy( meta->parent_block_id, ctx->parent_block_id, 32UL );
2058 0 : }
2059 :
2060 0 : dst += sizeof(fd_entry_batch_meta_t);
2061 0 : fd_entry_batch_header_t * header = (fd_entry_batch_header_t *)dst;
2062 0 : header->hashcnt_delta = hashcnt_delta;
2063 0 : fd_memcpy( header->hash, ctx->hash, 32UL );
2064 :
2065 0 : dst += sizeof(fd_entry_batch_header_t);
2066 0 : ulong payload_sz = 0UL;
2067 0 : ulong included_txn_cnt = 0UL;
2068 0 : for( ulong i=0UL; i<txn_cnt; i++ ) {
2069 0 : fd_txn_p_t * txn = (fd_txn_p_t *)(ctx->_txns + i*sizeof(fd_txn_p_t));
2070 0 : if( FD_UNLIKELY( !(txn->flags & FD_TXN_P_FLAGS_EXECUTE_SUCCESS) ) ) continue;
2071 :
2072 0 : fd_memcpy( dst, txn->payload, txn->payload_sz );
2073 0 : payload_sz += txn->payload_sz;
2074 0 : dst += txn->payload_sz;
2075 0 : included_txn_cnt++;
2076 0 : }
2077 0 : header->txn_cnt = included_txn_cnt;
2078 :
2079 : /* We always have credits to publish here, because we have a burst
2080 : value of 3 credits, and at most we will publish_tick() once and
2081 : then publish_became_leader() once, leaving one credit here to
2082 : publish the microblock. */
2083 0 : ulong tspub = (ulong)fd_frag_meta_ts_comp( fd_tickcount() );
2084 0 : ulong sz = sizeof(fd_entry_batch_meta_t)+sizeof(fd_entry_batch_header_t)+payload_sz;
2085 0 : ulong new_sig = fd_disco_poh_sig( slot, POH_PKT_TYPE_MICROBLOCK, 0UL );
2086 0 : fd_stem_publish( stem, ctx->shred_out->idx, new_sig, ctx->shred_out->chunk, sz, 0UL, 0UL, tspub );
2087 0 : ctx->shred_seq = stem->seqs[ ctx->shred_out->idx ];
2088 0 : ctx->shred_out->chunk = fd_dcache_compact_next( ctx->shred_out->chunk, sz, ctx->shred_out->chunk0, ctx->shred_out->wmark );
2089 :
2090 : /* Can only happen in low power mode. Refer to comment in publish_tick()
2091 : for more details */
2092 0 : if( FD_UNLIKELY( meta->block_complete && ctx->store_leader_bank_slot!=ULONG_MAX ) ) {
2093 0 : FD_TEST( ctx->store_leader_bank_slot==slot );
2094 0 : ctx->store_leader_bank = NULL;
2095 0 : ctx->store_leader_bank_slot = ULONG_MAX;
2096 0 : }
2097 0 : }
2098 :
2099 : static inline void
2100 : after_frag( fd_pohh_tile_t * ctx,
2101 : ulong in_idx,
2102 : ulong seq,
2103 : ulong sig,
2104 : ulong sz,
2105 : ulong tsorig,
2106 : ulong tspub,
2107 0 : fd_stem_context_t * stem ) {
2108 0 : (void)in_idx;
2109 0 : (void)seq;
2110 0 : (void)tsorig;
2111 0 : (void)tspub;
2112 :
2113 0 : if( FD_UNLIKELY( ctx->skip_frag ) ) return;
2114 :
2115 0 : if( FD_UNLIKELY( ctx->in_kind[ in_idx ]==IN_KIND_EPOCH ) ) {
2116 0 : fd_multi_epoch_leaders_stake_msg_fini( ctx->mleaders );
2117 : /* It might seem like we do not need to do state transitions in and
2118 : out of being the leader here, since leader schedule updates are
2119 : always one epoch in advance (whether we are leader or not would
2120 : never change for the currently executing slot) but this is not
2121 : true for new ledgers when the validator first boots. We will
2122 : likely be the leader in slot 1, and get notified of the leader
2123 : schedule for that slot while we are still in it.
2124 :
2125 : For safety we just handle both transitions, in and out, although
2126 : the only one possible should be into leader. */
2127 0 : ulong next_leader_slot_after_frag = next_leader_slot( ctx );
2128 :
2129 0 : int currently_leader = ctx->slot>=ctx->next_leader_slot;
2130 0 : int leader_after_frag = ctx->slot>=next_leader_slot_after_frag;
2131 :
2132 0 : FD_LOG_INFO(( "stake_update(before_leader=%lu,after_leader=%lu)",
2133 0 : ctx->next_leader_slot,
2134 0 : next_leader_slot_after_frag ));
2135 :
2136 0 : ctx->next_leader_slot = next_leader_slot_after_frag;
2137 0 : if( FD_UNLIKELY( currently_leader && !leader_after_frag ) ) {
2138 : /* Shouldn't ever happen, otherwise we need to do a state
2139 : transition out of being leader. */
2140 0 : FD_LOG_ERR(( "stake update caused us to no longer be leader in an active slot" ));
2141 0 : }
2142 :
2143 : /* Nothing to do if we transition into being leader, since it
2144 : will just get picked up by the regular tick loop. */
2145 0 : if( FD_UNLIKELY( !currently_leader && leader_after_frag ) ) {
2146 0 : publish_plugin_slot_start( ctx, next_leader_slot_after_frag, ctx->reset_slot );
2147 0 : }
2148 :
2149 0 : return;
2150 0 : }
2151 :
2152 0 : if( FD_UNLIKELY( !ctx->microblocks_lower_bound ) ) {
2153 0 : double tick_per_ns = fd_tempo_tick_per_ns( NULL );
2154 0 : fd_histf_sample( ctx->first_microblock_delay, (ulong)((double)(fd_log_wallclock()-ctx->reset_slot_start_ns)/tick_per_ns) );
2155 0 : }
2156 :
2157 0 : ulong target_slot = fd_disco_execle_sig_slot( sig );
2158 :
2159 0 : if( FD_UNLIKELY( target_slot!=ctx->next_leader_slot || target_slot!=ctx->slot ) ) {
2160 0 : FD_LOG_ERR(( "packed too early or late target_slot=%lu, current_slot=%lu. highwater_leader_slot=%lu",
2161 0 : target_slot, ctx->slot, ctx->highwater_leader_slot ));
2162 0 : }
2163 :
2164 0 : FD_TEST( ctx->current_leader_bank );
2165 0 : FD_TEST( ctx->microblocks_lower_bound<ctx->max_microblocks_per_slot );
2166 0 : ctx->microblocks_lower_bound += 1UL;
2167 :
2168 0 : ulong txn_cnt = (sz-sizeof(fd_microblock_trailer_t))/sizeof(fd_txn_p_t);
2169 0 : fd_txn_p_t * txns = (fd_txn_p_t *)(ctx->_txns);
2170 0 : ulong executed_txn_cnt = 0UL;
2171 0 : ulong cus_used = 0UL;
2172 0 : for( ulong i=0UL; i<txn_cnt; i++ ) {
2173 : /* It's important that we check if a transaction is included in the
2174 : block with FD_TXN_P_FLAGS_EXECUTE_SUCCESS since
2175 : actual_consumed_cus may have a nonzero value for excluded
2176 : transactions used for monitoring purposes */
2177 0 : if( FD_LIKELY( txns[ i ].flags & FD_TXN_P_FLAGS_EXECUTE_SUCCESS ) ) {
2178 0 : executed_txn_cnt++;
2179 0 : cus_used += txns[ i ].execle_cu.actual_consumed_cus;
2180 0 : }
2181 0 : }
2182 :
2183 : /* We don't publish transactions that fail to execute. If all the
2184 : transactions failed to execute, the microblock would be empty,
2185 : causing agave to think it's a tick and complain. Instead, we just
2186 : skip the microblock and don't hash or update the hashcnt. */
2187 0 : if( FD_UNLIKELY( !executed_txn_cnt ) ) return;
2188 :
2189 0 : uchar data[ 64 ];
2190 0 : fd_memcpy( data, ctx->hash, 32UL );
2191 0 : fd_memcpy( data+32UL, ctx->_microblock_trailer->hash, 32UL );
2192 0 : fd_sha256_hash( data, 64UL, ctx->hash );
2193 :
2194 0 : ctx->hashcnt++;
2195 0 : FD_TEST( ctx->hashcnt>ctx->last_hashcnt );
2196 0 : ulong hashcnt_delta = ctx->hashcnt - ctx->last_hashcnt;
2197 :
2198 : /* The hashing loop above will never leave us exactly one away from
2199 : crossing a tick boundary, so this increment will never cause the
2200 : current tick (or the slot) to change, except in low power mode
2201 : for development, in which case we do need to register the tick
2202 : with the leader bank. We don't need to publish the tick since
2203 : sending the microblock below is the publishing action. */
2204 0 : if( FD_UNLIKELY( !(ctx->hashcnt%ctx->hashcnt_per_slot ) ) ) {
2205 0 : ctx->slot++;
2206 0 : ctx->hashcnt = 0UL;
2207 0 : }
2208 :
2209 0 : ctx->last_slot = ctx->slot;
2210 0 : ctx->last_hashcnt = ctx->hashcnt;
2211 :
2212 0 : ctx->cus_used += cus_used;
2213 :
2214 0 : if( FD_UNLIKELY( !(ctx->hashcnt%ctx->hashcnt_per_tick ) ) ) {
2215 0 : fd_ext_poh_register_tick( ctx->current_leader_bank, ctx->hash );
2216 0 : }
2217 :
2218 0 : publish_microblock( ctx, stem, target_slot, hashcnt_delta, txn_cnt );
2219 :
2220 0 : if( FD_UNLIKELY( !(ctx->hashcnt%ctx->hashcnt_per_tick ) ) ) {
2221 0 : if( FD_UNLIKELY( ctx->slot>ctx->next_leader_slot ) ) {
2222 : /* We ticked while leader and are no longer leader... transition
2223 : the state machine. */
2224 0 : publish_plugin_slot_end( ctx, ctx->next_leader_slot, ctx->cus_used );
2225 :
2226 0 : no_longer_leader( ctx );
2227 :
2228 0 : if( FD_UNLIKELY( ctx->slot>=ctx->next_leader_slot ) ) {
2229 : /* We finished a leader slot, and are immediately leader for the
2230 : following slot... transition. */
2231 0 : publish_plugin_slot_start( ctx, ctx->next_leader_slot, ctx->next_leader_slot-1UL );
2232 0 : }
2233 0 : }
2234 0 : }
2235 0 : }
2236 :
2237 : static void
2238 : privileged_init( fd_topo_t const * topo,
2239 0 : fd_topo_tile_t const * tile ) {
2240 0 : void * scratch = fd_topo_obj_laddr( topo, tile->tile_obj_id );
2241 :
2242 0 : FD_SCRATCH_ALLOC_INIT( l, scratch );
2243 0 : fd_pohh_tile_t * ctx = FD_SCRATCH_ALLOC_APPEND( l, alignof( fd_pohh_tile_t ), sizeof( fd_pohh_tile_t ) );
2244 :
2245 0 : if( FD_UNLIKELY( !strcmp( tile->pohh.identity_key_path, "" ) ) )
2246 0 : FD_LOG_ERR(( "identity_key_path not set" ));
2247 :
2248 0 : const uchar * identity_key = fd_keyload_load( tile->pohh.identity_key_path, /* pubkey only: */ 1 );
2249 0 : fd_memcpy( ctx->identity_key.uc, identity_key, 32UL );
2250 :
2251 0 : ctx->bundle.enabled = tile->pohh.bundle.enabled;
2252 0 : if( FD_UNLIKELY( !tile->pohh.bundle.vote_account_path[0] ) ) {
2253 0 : ctx->bundle.enabled = 0;
2254 0 : }
2255 0 : if( FD_UNLIKELY( ctx->bundle.enabled ) ) {
2256 0 : if( FD_UNLIKELY( !fd_base58_decode_32( tile->pohh.bundle.vote_account_path, ctx->bundle.vote_account.uc ) ) ) {
2257 0 : const uchar * vote_key = fd_keyload_load( tile->pohh.bundle.vote_account_path, /* pubkey only: */ 1 );
2258 0 : fd_memcpy( ctx->bundle.vote_account.uc, vote_key, 32UL );
2259 0 : }
2260 0 : }
2261 0 : }
2262 :
2263 : /* The Agave client needs to communicate to the shred tile what
2264 : the shred version is on boot, but shred tile does not live in the
2265 : same address space, so have the PoH tile pass the value through
2266 : via. a shared memory ulong. */
2267 :
2268 : static volatile ulong * fd_shred_version;
2269 :
2270 : void
2271 0 : fd_ext_shred_set_shred_version( ulong shred_version ) {
2272 0 : while( FD_UNLIKELY( !fd_shred_version ) ) FD_SPIN_PAUSE();
2273 0 : *fd_shred_version = shred_version;
2274 0 : }
2275 :
2276 : void
2277 : fd_ext_poh_publish_gossip_vote( uchar * data,
2278 : ulong data_len,
2279 : uint source_ipv4,
2280 0 : uchar * pubkey ) {
2281 0 : (void)pubkey;
2282 0 : uchar txn_with_header[ FD_TPU_RAW_MTU ];
2283 0 : fd_txn_m_t * txnm = (fd_txn_m_t *)txn_with_header;
2284 0 : *txnm = (fd_txn_m_t) { 0UL };
2285 0 : txnm->payload_sz = (ushort)data_len;
2286 0 : txnm->source_ipv4 = source_ipv4;
2287 0 : txnm->source_tpu = FD_TXN_M_TPU_SOURCE_GOSSIP;
2288 0 : fd_memcpy(txn_with_header + sizeof(fd_txn_m_t), data, data_len);
2289 0 : poh_link_publish( &gossip_dedup, 1UL, txn_with_header, fd_txn_m_realized_footprint( txnm, 0, 0 ) );
2290 0 : }
2291 :
2292 : void
2293 : fd_ext_poh_publish_leader_schedule( uchar * data,
2294 0 : ulong data_len ) {
2295 0 : poh_link_publish( &stake_out, 2UL, data, data_len );
2296 0 : }
2297 :
2298 : void
2299 : fd_ext_poh_publish_cluster_info( uchar * data,
2300 0 : ulong data_len ) {
2301 0 : poh_link_publish( &crds_shred, 2UL, data, data_len );
2302 0 : }
2303 :
2304 : void
2305 0 : fd_ext_poh_publish_executed_txn( uchar const * data ) {
2306 0 : static int lock = 0;
2307 :
2308 : /* Need to lock since the link publisher is not concurrent, and replay
2309 : happens on a thread pool. */
2310 0 : for(;;) {
2311 0 : if( FD_LIKELY( FD_ATOMIC_CAS( &lock, 0, 1 )==0 ) ) break;
2312 0 : FD_SPIN_PAUSE();
2313 0 : }
2314 :
2315 0 : FD_COMPILER_MFENCE();
2316 0 : poh_link_publish( &executed_txn, 0UL, data, 64UL );
2317 0 : FD_COMPILER_MFENCE();
2318 :
2319 0 : FD_VOLATILE(lock) = 0;
2320 0 : }
2321 :
2322 : void
2323 : fd_ext_plugin_publish_replay_stage( ulong sig,
2324 : uchar * data,
2325 0 : ulong data_len ) {
2326 0 : poh_link_publish( &replay_plugin, sig, data, data_len );
2327 0 : }
2328 :
2329 : void
2330 : fd_ext_plugin_publish_genesis_hash( ulong sig,
2331 : uchar * data,
2332 0 : ulong data_len ) {
2333 0 : poh_link_publish( &replay_plugin, sig, data, data_len );
2334 0 : }
2335 :
2336 : void
2337 : fd_ext_plugin_publish_start_progress( ulong sig,
2338 : uchar * data,
2339 0 : ulong data_len ) {
2340 0 : poh_link_publish( &start_progress_plugin, sig, data, data_len );
2341 0 : }
2342 :
2343 : void
2344 : fd_ext_plugin_publish_vote_listener( ulong sig,
2345 : uchar * data,
2346 0 : ulong data_len ) {
2347 0 : poh_link_publish( &vote_listener_plugin, sig, data, data_len );
2348 0 : }
2349 :
2350 : void
2351 : fd_ext_plugin_publish_validator_info( ulong sig,
2352 : uchar * data,
2353 0 : ulong data_len ) {
2354 0 : poh_link_publish( &validator_info_plugin, sig, data, data_len );
2355 0 : }
2356 :
2357 : void
2358 : fd_ext_plugin_publish_periodic( ulong sig,
2359 : uchar * data,
2360 0 : ulong data_len ) {
2361 0 : poh_link_publish( &gossip_plugin, sig, data, data_len );
2362 0 : }
2363 :
2364 : void
2365 : fd_ext_resolv_publish_root_bank( uchar * data,
2366 0 : ulong data_len ) {
2367 0 : poh_link_publish( &replay_resolh, 0UL, data, data_len );
2368 0 : }
2369 :
2370 : void
2371 : fd_ext_resolv_publish_completed_blockhash( uchar * data,
2372 0 : ulong data_len ) {
2373 0 : poh_link_publish( &replay_resolh, 1UL, data, data_len );
2374 0 : }
2375 :
2376 : static inline fd_pohh_out_t
2377 : out1( fd_topo_t const * topo,
2378 : fd_topo_tile_t const * tile,
2379 0 : char const * name ) {
2380 0 : ulong idx = ULONG_MAX;
2381 :
2382 0 : for( ulong i=0UL; i<tile->out_cnt; i++ ) {
2383 0 : fd_topo_link_t const * link = &topo->links[ tile->out_link_id[ i ] ];
2384 0 : if( !strcmp( link->name, name ) ) {
2385 0 : if( FD_UNLIKELY( idx!=ULONG_MAX ) ) FD_LOG_ERR(( "tile %s:%lu had multiple output links named %s but expected one", tile->name, tile->kind_id, name ));
2386 0 : idx = i;
2387 0 : }
2388 0 : }
2389 :
2390 0 : if( FD_UNLIKELY( idx==ULONG_MAX ) ) FD_LOG_ERR(( "tile %s:%lu had no output link named %s", tile->name, tile->kind_id, name ));
2391 :
2392 0 : void * mem = topo->workspaces[ topo->objs[ topo->links[ tile->out_link_id[ idx ] ].dcache_obj_id ].wksp_id ].wksp;
2393 0 : ulong chunk0 = fd_dcache_compact_chunk0( mem, topo->links[ tile->out_link_id[ idx ] ].dcache );
2394 0 : ulong wmark = fd_dcache_compact_wmark ( mem, topo->links[ tile->out_link_id[ idx ] ].dcache, topo->links[ tile->out_link_id[ idx ] ].mtu );
2395 :
2396 0 : return (fd_pohh_out_t){ .idx = idx, .mem = mem, .chunk0 = chunk0, .wmark = wmark, .chunk = chunk0 };
2397 0 : }
2398 :
2399 : static void
2400 : unprivileged_init( fd_topo_t const * topo,
2401 0 : fd_topo_tile_t const * tile ) {
2402 0 : void * scratch = fd_topo_obj_laddr( topo, tile->tile_obj_id );
2403 :
2404 0 : FD_SCRATCH_ALLOC_INIT( l, scratch );
2405 0 : fd_pohh_tile_t * ctx = FD_SCRATCH_ALLOC_APPEND( l, alignof( fd_pohh_tile_t ), sizeof( fd_pohh_tile_t ) );
2406 0 : void * sha256 = FD_SCRATCH_ALLOC_APPEND( l, FD_SHA256_ALIGN, FD_SHA256_FOOTPRINT );
2407 :
2408 0 : #define NONNULL( x ) (__extension__({ \
2409 0 : __typeof__((x)) __x = (x); \
2410 0 : if( FD_UNLIKELY( !__x ) ) FD_LOG_ERR(( #x " was unexpectedly NULL" )); \
2411 0 : __x; }))
2412 :
2413 0 : ctx->mleaders = NONNULL( fd_multi_epoch_leaders_join( fd_multi_epoch_leaders_new( ctx->mleaders_mem ) ) );
2414 0 : ctx->sha256 = NONNULL( fd_sha256_join( fd_sha256_new( sha256 ) ) );
2415 0 : ctx->current_leader_bank = NULL;
2416 0 : ctx->pack_leader_bank = NULL;
2417 0 : ctx->store_leader_bank = NULL;
2418 0 : ctx->store_leader_bank_slot = ULONG_MAX;
2419 0 : ctx->signal_leader_change = NULL;
2420 :
2421 0 : ctx->shred_seq = ULONG_MAX;
2422 0 : ctx->halted_switching_key = 0;
2423 0 : ctx->keyswitch = fd_keyswitch_join( fd_topo_obj_laddr( topo, tile->id_keyswitch_obj_id ) );
2424 0 : FD_TEST( ctx->keyswitch );
2425 :
2426 0 : ctx->slot = 0UL;
2427 0 : ctx->hashcnt = 0UL;
2428 0 : ctx->last_hashcnt = 0UL;
2429 0 : ctx->highwater_leader_slot = ULONG_MAX;
2430 0 : ctx->next_leader_slot = ULONG_MAX;
2431 0 : ctx->reset_slot = ULONG_MAX;
2432 :
2433 0 : ctx->lagged_consecutive_leader_start = tile->pohh.lagged_consecutive_leader_start;
2434 0 : ctx->expect_sequential_leader_slot = ULONG_MAX;
2435 :
2436 0 : ctx->expect_pack_idx = 0U;
2437 0 : ctx->microblocks_lower_bound = 0UL;
2438 :
2439 0 : ctx->max_active_descendant = 0UL;
2440 :
2441 0 : if( FD_UNLIKELY( ctx->bundle.enabled ) ) {
2442 0 : NONNULL( fd_bundle_crank_gen_init( ctx->bundle.gen, (fd_acct_addr_t const *)tile->pohh.bundle.tip_distribution_program_addr,
2443 0 : (fd_acct_addr_t const *)tile->pohh.bundle.tip_payment_program_addr,
2444 0 : (fd_acct_addr_t const *)ctx->bundle.vote_account.uc,
2445 0 : (fd_acct_addr_t const *)ctx->bundle.vote_account.uc, "NAN", 0UL ) ); /* last three arguments are properly bogus */
2446 0 : }
2447 :
2448 0 : ulong pohh_shred_obj_id = fd_pod_query_ulong( topo->props, "pohh_shred", ULONG_MAX );
2449 0 : FD_TEST( pohh_shred_obj_id!=ULONG_MAX );
2450 :
2451 0 : fd_shred_version = fd_fseq_join( fd_topo_obj_laddr( topo, pohh_shred_obj_id ) );
2452 0 : FD_TEST( fd_shred_version );
2453 :
2454 0 : poh_link_init( &gossip_dedup, topo, tile, out1( topo, tile, "gossip_dedup" ).idx );
2455 0 : poh_link_init( &stake_out, topo, tile, out1( topo, tile, "stake_out" ).idx );
2456 0 : poh_link_init( &crds_shred, topo, tile, out1( topo, tile, "crds_shred" ).idx );
2457 0 : poh_link_init( &replay_resolh, topo, tile, out1( topo, tile, "replay_resol" ).idx );
2458 0 : poh_link_init( &executed_txn, topo, tile, out1( topo, tile, "executed_txn" ).idx );
2459 :
2460 0 : if( FD_LIKELY( tile->pohh.plugins_enabled ) ) {
2461 0 : poh_link_init( &replay_plugin, topo, tile, out1( topo, tile, "replay_plugi" ).idx );
2462 0 : poh_link_init( &gossip_plugin, topo, tile, out1( topo, tile, "gossip_plugi" ).idx );
2463 0 : poh_link_init( &start_progress_plugin, topo, tile, out1( topo, tile, "startp_plugi" ).idx );
2464 0 : poh_link_init( &vote_listener_plugin, topo, tile, out1( topo, tile, "votel_plugin" ).idx );
2465 0 : poh_link_init( &validator_info_plugin, topo, tile, out1( topo, tile, "valcfg_plugi" ).idx );
2466 0 : } else {
2467 : /* Mark these mcaches as "available", so the system boots, but the
2468 : memory is not set so nothing will actually get published via.
2469 : the links. */
2470 0 : FD_COMPILER_MFENCE();
2471 0 : replay_plugin.mcache = (fd_frag_meta_t*)1;
2472 0 : gossip_plugin.mcache = (fd_frag_meta_t*)1;
2473 0 : start_progress_plugin.mcache = (fd_frag_meta_t*)1;
2474 0 : vote_listener_plugin.mcache = (fd_frag_meta_t*)1;
2475 0 : validator_info_plugin.mcache = (fd_frag_meta_t*)1;
2476 0 : FD_COMPILER_MFENCE();
2477 0 : }
2478 :
2479 0 : FD_LOG_INFO(( "PoH waiting to be initialized by Agave client... %lu %lu", fd_poh_waiting_lock, fd_poh_returned_lock ));
2480 0 : FD_VOLATILE( fd_pohh_global_ctx ) = ctx;
2481 0 : FD_COMPILER_MFENCE();
2482 0 : for(;;) {
2483 0 : if( FD_LIKELY( FD_VOLATILE_CONST( fd_poh_waiting_lock ) ) ) break;
2484 0 : FD_SPIN_PAUSE();
2485 0 : }
2486 0 : FD_VOLATILE( fd_poh_waiting_lock ) = 0UL;
2487 0 : FD_VOLATILE( fd_poh_returned_lock ) = 1UL;
2488 0 : FD_COMPILER_MFENCE();
2489 0 : for(;;) {
2490 0 : if( FD_UNLIKELY( !FD_VOLATILE_CONST( fd_poh_returned_lock ) ) ) break;
2491 0 : FD_SPIN_PAUSE();
2492 0 : }
2493 0 : FD_COMPILER_MFENCE();
2494 :
2495 0 : if( FD_UNLIKELY( ctx->reset_slot==ULONG_MAX ) ) FD_LOG_ERR(( "PoH was not initialized by Agave client" ));
2496 :
2497 0 : fd_histf_join( fd_histf_new( ctx->begin_leader_delay, FD_MHIST_SECONDS_MIN( POHH, BEGIN_LEADER_DELAY_SECONDS ),
2498 0 : FD_MHIST_SECONDS_MAX( POHH, BEGIN_LEADER_DELAY_SECONDS ) ) );
2499 0 : fd_histf_join( fd_histf_new( ctx->first_microblock_delay, FD_MHIST_SECONDS_MIN( POHH, FIRST_MICROBLOCK_DELAY_SECONDS ),
2500 0 : FD_MHIST_SECONDS_MAX( POHH, FIRST_MICROBLOCK_DELAY_SECONDS ) ) );
2501 0 : fd_histf_join( fd_histf_new( ctx->slot_done_delay, FD_MHIST_SECONDS_MIN( POHH, SLOT_DONE_DELAY_SECONDS ),
2502 0 : FD_MHIST_SECONDS_MAX( POHH, SLOT_DONE_DELAY_SECONDS ) ) );
2503 :
2504 0 : fd_histf_join( fd_histf_new( ctx->bundle_init_delay, FD_MHIST_SECONDS_MIN( POHH, BUNDLE_INITIALIZE_DELAY_SECONDS ),
2505 0 : FD_MHIST_SECONDS_MAX( POHH, BUNDLE_INITIALIZE_DELAY_SECONDS ) ) );
2506 :
2507 0 : for( ulong i=0UL; i<tile->in_cnt; i++ ) {
2508 0 : fd_topo_link_t const * link = &topo->links[ tile->in_link_id[ i ] ];
2509 0 : fd_topo_wksp_t const * link_wksp = &topo->workspaces[ topo->objs[ link->dcache_obj_id ].wksp_id ];
2510 :
2511 0 : ctx->in[ i ].mem = link_wksp->wksp;
2512 0 : ctx->in[ i ].chunk0 = fd_dcache_compact_chunk0( ctx->in[ i ].mem, link->dcache );
2513 0 : ctx->in[ i ].wmark = fd_dcache_compact_wmark ( ctx->in[ i ].mem, link->dcache, link->mtu );
2514 :
2515 0 : if( !strcmp( link->name, "stake_out" ) ) {
2516 0 : ctx->in_kind[ i ] = IN_KIND_EPOCH;
2517 0 : } else if( !strcmp( link->name, "pack_pohh" ) ) {
2518 0 : ctx->in_kind[ i ] = IN_KIND_PACK;
2519 0 : } else if( !strcmp( link->name, "bank_pohh" ) ) {
2520 0 : ctx->in_kind[ i ] = IN_KIND_BANK;
2521 0 : } else {
2522 0 : FD_LOG_ERR(( "unexpected input link name %s", link->name ));
2523 0 : }
2524 0 : }
2525 :
2526 0 : *ctx->shred_out = out1( topo, tile, "pohh_shred" );
2527 0 : *ctx->pack_out = out1( topo, tile, "pohh_pack" );
2528 0 : ctx->plugin_out->mem = NULL;
2529 0 : if( FD_LIKELY( tile->pohh.plugins_enabled ) ) {
2530 0 : *ctx->plugin_out = out1( topo, tile, "pohh_plugin" );
2531 0 : }
2532 :
2533 0 : ctx->shred_epoch_msg_avail = 0UL;
2534 0 : for( ulong i=0UL; i<FD_SHRED_FEATURES_ACTIVATION_SLOT_CNT; i++ )
2535 0 : ctx->shred_epoch_msg->features_activation.slots[i] = FD_SHRED_FEATURES_ACTIVATION_SLOT_DISABLED;
2536 :
2537 0 : ctx->shred_epoch_msg->slot_limits.prev_max_shred_idx = FD_SHRED_BLK_MAX;
2538 0 : ctx->shred_epoch_msg->slot_limits.current_max_shred_idx = FD_SHRED_BLK_MAX;
2539 0 : ctx->shred_epoch_msg->slot_limits.next_max_shred_idx = FD_SHRED_BLK_MAX;
2540 0 : ctx->shred_epoch_msg->slot_limits.current_start_slot = 0UL;
2541 0 : ctx->shred_epoch_msg->slot_limits.next_start_slot = ULONG_MAX;
2542 :
2543 0 : ulong scratch_top = FD_SCRATCH_ALLOC_FINI( l, scratch_align() );
2544 0 : if( FD_UNLIKELY( scratch_top > (ulong)scratch + scratch_footprint( tile ) ) )
2545 0 : FD_LOG_ERR(( "scratch overflow %lu %lu %lu", scratch_top - (ulong)scratch - scratch_footprint( tile ), scratch_top, (ulong)scratch + scratch_footprint( tile ) ));
2546 0 : }
2547 :
2548 : /* One tick, one microblock, one plugin slot end, one plugin slot start,
2549 : one leader update, one features activation, and one leader bank
2550 : handoff. */
2551 0 : #define STEM_BURST (7UL)
2552 :
2553 : /* See explanation in fd_pack */
2554 0 : #define STEM_LAZY (128L*3000L)
2555 :
2556 0 : #define STEM_CALLBACK_CONTEXT_TYPE fd_pohh_tile_t
2557 0 : #define STEM_CALLBACK_CONTEXT_ALIGN alignof(fd_pohh_tile_t)
2558 :
2559 0 : #define STEM_CALLBACK_DURING_HOUSEKEEPING during_housekeeping
2560 0 : #define STEM_CALLBACK_METRICS_WRITE metrics_write
2561 0 : #define STEM_CALLBACK_AFTER_CREDIT after_credit
2562 0 : #define STEM_CALLBACK_BEFORE_FRAG before_frag
2563 0 : #define STEM_CALLBACK_DURING_FRAG during_frag
2564 0 : #define STEM_CALLBACK_AFTER_FRAG after_frag
2565 :
2566 : #include "../../disco/stem/fd_stem.c"
2567 :
2568 : fd_topo_run_tile_t fd_tile_pohh = {
2569 : .name = "pohh",
2570 : .populate_allowed_seccomp = NULL,
2571 : .populate_allowed_fds = NULL,
2572 : .scratch_align = scratch_align,
2573 : .scratch_footprint = scratch_footprint,
2574 : .privileged_init = privileged_init,
2575 : .unprivileged_init = unprivileged_init,
2576 : .run = stem_run,
2577 : };
|
