Line data Source code
1 : #ifndef HEADER_fd_src_choreo_tower_fd_tower_h
2 : #define HEADER_fd_src_choreo_tower_fd_tower_h
3 :
4 :
5 : /* Tower implements Solana's TowerBFT algorithm.
6 :
7 : Starting with some intuition, the goal of TowerBFT is for a
8 : supermajority of the validator cluster to pick the same fork. As we
9 : know, we receive and execute blocks during replay.
10 :
11 : /-- 3-- 4 (A)
12 : 1-- 2
13 : \-- 5 (B)
14 :
15 : Here, is a diagram of a fork. The leader for slot 5 decided to build
16 : off slot 2, rather than slot 4. This can happen for various reasons,
17 : for example network propagation delay. We now have two possible forks
18 : labeled A and B. The consensus algorithm has to pick one of them.
19 :
20 : So, how does the consensus algorithm pick? As detailed in fd_ghost.h,
21 : we pick the fork based on the most stake from votes, called the
22 : “heaviest”. Validators vote for blocks during replay, and
23 : simultaneously use other validator’s votes to determine which block
24 : to vote for. This encourages convergence, because as one fork gathers
25 : more votes, more and more votes pile on, solidifying its position as
26 : the heaviest fork.
27 :
28 : /-- 3-- 4 (10%)
29 : 1-- 2
30 : \-- 5 (9%)
31 :
32 : However, network propagation delay of votes can lead us to think one
33 : fork is heaviest, before observing new votes that indicate another
34 : fork is heavier. So our consensus algorithm also needs to support
35 : switching.
36 :
37 : /-- 3-- 4 (10%)
38 : 1-- 2
39 : \-- 5 (15%)
40 :
41 : At the same time we don’t want excessive switching. The more often
42 : validators switch, the more difficult it will be to achieve that pile
43 : on effect I just described.
44 :
45 : Note that to switch forks, you need to rollback a given slot and its
46 : descendants on that fork. In the example above, to switch to 1, 2, 5,
47 : we need to rollback 3 and 4. The consensus algorithm makes it more
48 : costly the further you want to rollback a fork. Here, I’ve added a
49 : column lockout, which doubles for every additional slot you want to
50 : rollback.
51 :
52 : Eventually you have traversed far enough down a fork, that the
53 : lockout is so great it is infeasible to imagine it ever rolling back
54 : in practice. So you can make that fork permanent or “commit” it. Once
55 : all validators do this, the blockchain now just has a single fork.
56 :
57 : Armed with some intuition, now let’s begin defining some terminology.
58 : Here is a diagram of a validator's "vote tower":
59 :
60 : slot | confirmation count (conf)
61 : --------------------------------
62 : 4 | 1
63 : 3 | 2
64 : 2 | 3
65 : 1 | 4
66 :
67 : It is a stack structure in which each element is a vote. The vote
68 : slot column indicates which slots the validator has voted for,
69 : ordered from most to least recent.
70 :
71 : The confirmation count column indicates how many consecutive votes on
72 : the same fork have been pushed on top of that vote. You are
73 : confirming your own votes for a fork every time you vote on top of
74 : the same fork.
75 :
76 : Two related concepts to confirmation count are lockout and expiration
77 : slot. Lockout equals 2 to the power of confirmation count. Every time
78 : we “confirm” a vote by voting on top of it, we double the lockout.
79 : The expiration slot is the sum of vote slot and lockout, so it also
80 : increases when lockouts double. It represents which slot the vote
81 : will expire. When a vote expires, it is popped from the top of the
82 : tower. An important Tower rule is that a validator cannot vote for a
83 : different fork from a given vote slot, until reaching the expiration
84 : slot for that vote slot. To summarize, the further a validator wants
85 : to rollback their fork (or vote slots) the longer the validator needs
86 : to wait without voting (in slot time).
87 :
88 : Here is the same tower, fully-expanded to include all the fields:
89 :
90 : slot | conf | lockout | expiration
91 : ----------------------------------
92 : 4 | 1 | 2 | 6
93 : 3 | 2 | 4 | 7
94 : 2 | 3 | 8 | 10
95 : 1 | 4 | 16 | 17
96 :
97 : Based on this tower, the validator is locked out from voting for any
98 : slot <= 6 that is on a different fork than slot 4. I’d like to
99 : emphasize that the expiration is with respect to the vote slot, and
100 : is _not_ related to the Proof-of-History slot or what the
101 : quote-unquote current slot is. So even if the current slot is now 7,
102 : the validator can’t go back and vote for slot 5, if slot 5 were on a
103 : different fork than 4. The earliest valid vote slot this validator
104 : could submit for a different fork from 4 would be slot 7 or later.
105 :
106 : Next let’s look at how the tower makes state transitions. Here we
107 : have the previous example tower, with a before-and-after view with
108 : respect to a vote for slot 9:
109 :
110 : slot | conf
111 : (before) -----------
112 : 4 | 1
113 : 3 | 2
114 : 2 | 3
115 : 1 | 4
116 :
117 : slot | conf
118 : (after) -----------
119 : 9 | 1
120 : 2 | 3
121 : 1 | 4
122 :
123 : As you can see, we added a vote for slot 9 to the top of the tower.
124 : But we also removed the votes for slot 4 and slot 3. What happened?
125 : This is an example of vote expiry in action. When we voted for slot
126 : 9, this exceeded the expirations of vote slots 4 and 3, which were 6
127 : and 7 respectively. This action of voting triggered the popping of
128 : the expired votes from the top of the tower.
129 :
130 : Next, we add a vote for slot 11:
131 :
132 : slot | conf
133 : (before) -----------
134 : 9 | 1
135 : 2 | 3
136 : 1 | 4
137 :
138 : slot | conf
139 : (after) -----------
140 : 11 | 1
141 : 9 | 2
142 : 2 | 3
143 : 1 | 4
144 :
145 : The next vote for slot 11 doesn’t involve expirations, so we just add
146 : it to the top of the tower. Also, here is an important property of
147 : lockouts. Note that the lockout for vote slot 9 doubled (ie. the
148 : confirmation count increased by 1) but the lockouts of vote slots 2
149 : and 1 remained unchanged.
150 :
151 : The reason for this is confirmation counts only increase when they
152 : are consecutive in the vote tower. Because 4 and 3 were expired
153 : previously by the vote for 9, that consecutive property was broken.
154 : In this case, the vote for slot 11 is only consecutive with slot 9,
155 : but not 2 and 1. Specifically, there is a gap in the before-tower at
156 : confirmation count 2.
157 :
158 : In the after-tower, all the votes are again consecutive (confirmation
159 : counts 1, 2, 3, 4 are all accounted for), so the next vote will
160 : result in all lockouts doubling as long as it doesn’t result in more
161 : expirations.
162 :
163 : One other thing I’d like to point out about this vote for slot 11.
164 : Even though 11 exceeds the expiration slot of vote slot 2, which is
165 : 10, voting for 11 did not expire the vote for 2. This is because
166 : expiration happens top-down and contiguously. Because vote slot 9 was
167 : not expired, we do not proceed with expiring 2.
168 :
169 : In the Tower rules, once a vote reaches a max lockout of 32, it is
170 : considered rooted and it is popped from the bottom of the tower. Here
171 : is an example:
172 :
173 : slot | conf
174 : (before) -----------
175 : 50 | 1
176 : ... | ... <- 29 votes elided
177 : 1 | 4
178 :
179 : slot | conf
180 : (after) -----------
181 : 53 | 1
182 : ... | ... <- 29 votes elided
183 : 1 | 4
184 :
185 :
186 : So the tower is really a double-ended queue rather than a stack.
187 :
188 : Rooting has implications beyond the Tower. It's what we use to prune
189 : our state. Every time tower makes a new root slot, we prune any old
190 : state that does not originate from that new root slot. Our blockstore
191 : will discard blocks below that root, our forks structure will discard
192 : stale banks, funk (which is our accounts database) will discard stale
193 : transactions (which in turn track modifications to accounts), and
194 : ghost (which is our fork select tree) will discard stale nodes
195 : tracking stake percentages. We call this operation publishing.
196 :
197 : Note that the vote slots are not necessarily consecutive. Here I
198 : elided the votes sandwiched between the newest and oldest votes for
199 : brevity.
200 :
201 : Next, let’s go over three additional tower checks. These three checks
202 : further reinforce the consensus algorithm we established with
203 : intuition, in this case getting a supermajority (ie. 2/3) of stake to
204 : converge on a fork.
205 :
206 : The first is the threshold check. The threshold check makes sure at
207 : least 2/3 of stake has voted for the same fork as the vote at depth 8
208 : in our tower. Essentially, this guards our tower from getting too out
209 : of sync with the rest of the cluster. If we get too out of sync we
210 : can’t vote for a long time, because we had to rollback a vote we had
211 : already confirmed many times and had a large lockout. This might
212 : otherwise happen as the result of a network partition where we can
213 : only communicate with a subset of stake.
214 :
215 : Next is the lockout check. We went in detail on this earlier when
216 : going through the lockout and expiration slot, and as before, the
217 : rule is we can only vote on a slot for a different fork from a
218 : previous vote, after that vote’s expiration slot.
219 :
220 : Given this fork and tower from earlier:
221 :
222 : /-- 3-- 4
223 : 1-- 2
224 : \-- 5
225 :
226 : slot | conf
227 : -----------
228 : 4 | 1
229 : 3 | 2
230 : 2 | 3
231 : 1 | 4
232 :
233 : You’re locked out from voting for 5 because it’s on a different fork
234 : from 4 and the expiration slot of your previous vote for 4 is 6.
235 :
236 : However, if we introduce a new slot 9:
237 :
238 : /-- 3-- 4
239 : 1-- 2
240 : \-- 5-- 9
241 :
242 : slot | conf
243 : -----------
244 : 9 | 1
245 : 2 | 3
246 : 1 | 4
247 :
248 : Here the new Slot 9 descends from 5, and exceeds vote slot 2’s
249 : expiration slot of 6 unlike 5.
250 :
251 : After your lockout expires, the tower rules allow you to vote for
252 : descendants of the fork slot you wanted to switch to in the first
253 : place (here, 9 descending from 5). So we eventually switch to the fork
254 : we wanted, by voting for 9 and expiring 3 and 4.
255 :
256 : Importantly, notice that the fork slots and vote slots are not exactly
257 : 1-to-1. While conceptually our tower is voting for the fork 1, 2, 5,
258 : 9, the vote for 5 is only implied. Our tower votes themselves still
259 : can’t include 5 due to lockout.
260 :
261 : Finally, the switch check. The switch check is used to safeguard
262 : optimistic confirmation. I won’t go into detail on optimistic
263 : confirmation, but in a nutshell it enables a fast-fork compared to
264 : rooting for a client to have high confidence a slot will eventually
265 : get rooted.
266 :
267 : The switch check is additional to the lockout check. Before switching
268 : forks, we need to make sure at least 38% of stake has voted for a
269 : different fork than our own. Different fork is defined by finding the
270 : greatest common ancestor of our last voted fork slot and the slot we
271 : want to switch to. Any forks descending from the greatest common
272 : ancestor (which I will subsequently call the GCA) that are not our
273 : own fork are counted towards the switch check stake.
274 :
275 : Here we visualize the switch check:
276 :
277 : /-- 7
278 : /-- 3-- 4
279 : 1-- 2 -- 6
280 : \-- 5-- 9
281 :
282 : First, we find the GCA of 4 and 9 which is 2. Then we look at all the
283 : descendants of the GCA that do not share a fork with us, and make
284 : sure their stake sums to more than 38%.
285 :
286 : I’d like to highlight that 7 here is not counted towards the switch
287 : proof, even though it is on a different fork from 4. This is because
288 : it’s on the same fork relative to the GCA.
289 :
290 : Now let’s switch gears from theory back to practice. What does it
291 : mean to send a vote?
292 :
293 : As a validator, you aren’t sending individual tower votes. Rather,
294 : you are sending your entire updated tower to the cluster every time.
295 : Essentially, the validator is continuously syncing their local tower
296 : with the cluster. That tower state is then stored inside a vote
297 : account, like any other state on Solana.
298 :
299 : On the flip side, we also must sync the other way: from cluster to
300 : local. If we have previously voted, we need to make sure we’re
301 : starting from where the cluster thinks we last left off. Conveniently
302 : Funk, our accounts database, stores all the vote accounts including
303 : our own, so on bootstrap we simply load in our vote account state
304 : itself to to initialize our own local view of the tower.
305 :
306 : *Additional Considerations*
307 :
308 : What's the difference between TowerBFT and the Vote Program?
309 :
310 : - TowerBFT runs on the sending side against our own tower ("local"
311 : view). It updates the tower with votes based on the algorithm
312 : detailed above. Importantly, TowerBFT has a view of all forks, and
313 : the validator makes a voting decision based on all forks.
314 :
315 : - The Vote Program runs on the receiving side against others' towers
316 : ("cluster" view). It checks that invariants about TowerBFT are
317 : maintained on votes received from the cluster. These checks are
318 : comparatively superficial to all the rules in tower. Furthermore,
319 : given it is a native program, the Vote Program only has access to
320 : the limited state programs are subject to. Specifically, it only
321 : has a view of the current fork it is executing on. It can't
322 : determine things like how much stake is allocated to other forks.
323 :
324 : What happens if our tower is out of sync with the cluster
325 : supermajority root (SMR)?
326 :
327 : - We detect this by seeing that our latest vote no longer descends
328 : from the SMR. Consider 2 cases:
329 :
330 : 1. We are stuck on a minority fork. In this case, we will observe
331 : that our latest vote slot > SMR, but its ancestry does not
332 : connect back to the SMR. This can happen if we get locked out
333 : for a long time by voting for (and confirming) a minority fork.
334 :
335 : 2. We have a latest vote slot that is older than the SMR, meaning
336 : the cluster has rooted past our latest vote. This can happen
337 : when we don't vote for a while (e.g. this validator was not
338 : running or we were stuck waiting for lockouts.)
339 : */
340 :
341 : #include "../../flamenco/runtime/fd_blockstore.h"
342 : #include "../fd_choreo_base.h"
343 : #include "../forks/fd_forks.h"
344 : #include "../ghost/fd_ghost.h"
345 :
346 : #define FD_TOWER_EQV_SAFE ( 0.52 )
347 : #define FD_TOWER_OPT_CONF ( 2.0 / 3.0 )
348 0 : #define FD_TOWER_VOTE_MAX ( 32UL )
349 :
350 : /* FD_TOWER_USE_HANDHOLDING: Define this to non-zero at compile time
351 : to turn on additional runtime checks and logging. */
352 :
353 : #ifndef FD_TOWER_USE_HANDHOLDING
354 : #define FD_TOWER_USE_HANDHOLDING 1
355 : #endif
356 :
357 : struct fd_tower_vote {
358 : ulong slot; /* vote slot */
359 : ulong conf; /* confirmation count */
360 : };
361 : typedef struct fd_tower_vote fd_tower_vote_t;
362 :
363 : #define DEQUE_NAME fd_tower_votes
364 0 : #define DEQUE_T fd_tower_vote_t
365 0 : #define DEQUE_MAX FD_TOWER_VOTE_MAX
366 : #include "../../util/tmpl/fd_deque.c"
367 :
368 : struct fd_tower_vote_acc {
369 : fd_pubkey_t const * addr;
370 : ulong stake;
371 : };
372 : typedef struct fd_tower_vote_acc fd_tower_vote_acc_t;
373 :
374 : #define DEQUE_NAME fd_tower_vote_accs
375 : #define DEQUE_T fd_tower_vote_acc_t
376 0 : #define DEQUE_MAX FD_VOTER_MAX
377 : #include "../../util/tmpl/fd_deque.c"
378 :
379 : /* fd_tower implements the TowerBFT algorithm and related consensus
380 : rules. */
381 :
382 : /* clang-format off */
383 : struct __attribute__((aligned(128UL))) fd_tower {
384 :
385 : /* The votes currently in the tower, ordered from latest to earliest
386 : vote slot (lowest to highest confirmation count). */
387 :
388 : fd_tower_vote_t * votes;
389 :
390 : /* The root is the most recent vote in the tower to reach max lockout
391 : (ie. confirmation count 32). It is no longer present in the tower
392 : votes themselves. */
393 :
394 : ulong root; /* FIXME wire with fseq */
395 :
396 : /* Vote accounts in the current epoch.
397 :
398 : Lifetimes of the vote account addresses (pubkeys) are valid for the
399 : epoch (the pubkey memory is owned by the epoch bank.) */
400 :
401 : fd_tower_vote_acc_t * vote_accs;
402 :
403 : /* Total amount of stake in the current epoch. */
404 :
405 : ulong total_stake;
406 :
407 : /* smr is a non-NULL pointer to an fseq that always contains the
408 : highest observed smr. This value is initialized by replay tile.
409 :
410 : Do not read or modify outside the fseq API. */
411 :
412 : ulong * smr;
413 : };
414 : typedef struct fd_tower fd_tower_t;
415 :
416 : /* fd_tower_{align,footprint} return the required alignment and
417 : footprint of a memory region suitable for use as tower. align is
418 : double cache line to mitigate false sharing. */
419 :
420 : FD_FN_CONST static inline ulong
421 0 : fd_tower_align( void ) {
422 0 : return alignof(fd_tower_t);
423 0 : }
424 :
425 : FD_FN_CONST static inline ulong
426 0 : fd_tower_footprint( void ) {
427 0 : return FD_LAYOUT_FINI(
428 0 : FD_LAYOUT_APPEND(
429 0 : FD_LAYOUT_APPEND(
430 0 : FD_LAYOUT_APPEND(
431 0 : FD_LAYOUT_INIT,
432 0 : alignof(fd_tower_t), sizeof(fd_tower_t) ),
433 0 : fd_tower_votes_align(), fd_tower_votes_footprint() ),
434 0 : fd_tower_vote_accs_align(), fd_tower_vote_accs_footprint() ),
435 0 : alignof(fd_tower_t) );
436 0 : }
437 : /* clang-format on */
438 :
439 : /* fd_tower_new formats an unused memory region for use as a tower. mem
440 : is a non-NULL pointer to this region in the local address space with
441 : the required footprint and alignment. */
442 :
443 : void *
444 : fd_tower_new( void * mem );
445 :
446 : /* fd_tower_join joins the caller to the tower. tower points to the
447 : first byte of the memory region backing the tower in the caller's
448 : address space.
449 :
450 : Returns a pointer in the local address space to tower on success. */
451 :
452 : fd_tower_t *
453 : fd_tower_join( void * tower );
454 :
455 : /* fd_tower_leave leaves a current local join. Returns a pointer to the
456 : underlying shared memory region on success and NULL on failure (logs
457 : details). Reasons for failure include tower is NULL. */
458 :
459 : void *
460 : fd_tower_leave( fd_tower_t const * tower );
461 :
462 : /* fd_tower_delete unformats a memory region used as a tower. Assumes
463 : only the local process is joined to the region. Returns a pointer to
464 : the underlying shared memory region or NULL if used obviously in
465 : error (e.g. tower is obviously not a tower ... logs details). The
466 : ownership of the memory region is transferred to the caller. */
467 :
468 : void *
469 : fd_tower_delete( void * tower );
470 :
471 : /* fd_tower_init initializes a tower. Assumes tower is a valid local
472 : join and no other processes are joined. root is the initial root
473 : that tower will use. This is the snapshot slot if booting from a
474 : snapshot, genesis slot otherwise.
475 :
476 : In general, this should be called by the same process that formatted
477 : tower's memory, ie. the caller of fd_tower_new. */
478 : void
479 : fd_tower_init( fd_tower_t * tower,
480 : fd_pubkey_t const * vote_acc_addr,
481 : fd_acc_mgr_t * acc_mgr,
482 : fd_exec_epoch_ctx_t const * epoch_ctx,
483 : fd_fork_t const * fork,
484 : ulong * smr );
485 :
486 : /* fd_tower_lockout_check checks if we are locked out from voting for
487 : fork. Returns 1 if we can vote for fork without violating lockout, 0
488 : otherwise.
489 :
490 : After voting for a slot n, we are locked out for 2^k slots, where k
491 : is the confirmation count of that vote. Once locked out, we cannot
492 : vote for a different fork until that previously-voted fork expires at
493 : slot n+2^k. This implies the earliest slot in which we can switch
494 : from the previously-voted fork is (n+2^k)+1.
495 :
496 : In the case of the tower, every vote has its own expiration slot
497 : depending on confirmations. The confirmation count is the max number
498 : of consecutive votes that have been pushed on top of the vote, and
499 : not necessarily its current height in the tower.
500 :
501 : For example, the following is a diagram of a tower pushing and
502 : popping with each vote:
503 :
504 :
505 : slot | confirmation count
506 : -----|-------------------
507 : 4 | 1 <- vote
508 : 3 | 2
509 : 2 | 3
510 : 1 | 4
511 :
512 :
513 : slot | confirmation count
514 : -----|-------------------
515 : 9 | 1 <- vote
516 : 2 | 3
517 : 1 | 4
518 :
519 :
520 : slot | confirmation count
521 : -----|-------------------
522 : 10 | 1 <- vote
523 : 9 | 2
524 : 2 | 3
525 : 1 | 4
526 :
527 :
528 : slot | confirmation count
529 : -----|-------------------
530 : 11 | 1 <- vote
531 : 10 | 2
532 : 9 | 3
533 : 2 | 4
534 : 1 | 5
535 :
536 :
537 : slot | confirmation count
538 : -----|-------------------
539 : 18 | 1 <- vote
540 : 2 | 4
541 : 1 | 5
542 :
543 :
544 : In the final tower, note the gap in confirmation counts between slot
545 : 18 and slot 2, even though slot 18 is directly above slot 2. */
546 :
547 : int
548 : fd_tower_lockout_check( fd_tower_t const * tower,
549 : fd_fork_t const * fork,
550 : fd_ghost_t const * ghost );
551 :
552 : /* fd_tower_switch_check checks if we can switch to fork. Returns 1 if
553 : we can switch, 0 otherwise.
554 :
555 : The switch rule is based on the percentage of validators whose: 1.
556 : last vote slot is for fork and 2. lockout prevents them from voting
557 : for our last vote slot.
558 :
559 : A validator is locked out from voting for our last vote slot when
560 : their last vote slot is on a different fork, and that vote's
561 : expiration slot > our last vote slot.
562 :
563 : The following pseudocode describes the algorithm:
564 :
565 : ```
566 : find the greatest common ancestor (gca) of our fork and fork
567 : for all validators v
568 : if v's latest vote is for fork
569 : add v's stake to switch stake
570 :
571 :
572 : for all validators v
573 : if v's locked out[1] from voting for our latest vote slot
574 : add v's stake to switch stake
575 : return switch stake >= FD_TOWER_SWITCH_PCT
576 : ```
577 :
578 :
579 : The switch check is used to safeguard optimistic confirmation.
580 : Invariant: FD_TOWER_OPT_CONF_PCT + FD_TOWER_SWITCH_PCT >= 1. */
581 :
582 : int
583 : fd_tower_switch_check( fd_tower_t const * tower, fd_fork_t const * fork, fd_ghost_t const * ghost );
584 :
585 : /* fd_tower_threshold_check checks if we pass the threshold required to
586 : continue voting along the same fork as our last vote. Returns 1 if
587 : we pass the threshold check, 0 otherwise.
588 :
589 : The following psuedocode describes the algorithm:
590 :
591 : ```
592 : for all vote accounts on the current fork
593 :
594 : simulate that the validator has voted on the current slot (the
595 : fork head)
596 :
597 : pop all votes expired by that simulated vote
598 :
599 : if validator's latest tower vote after expiry >= our threshold
600 : slot ie. our vote from FD_TOWER_THRESHOLD_DEPTH back (after
601 : simulating a vote on our own tower the same way)
602 :
603 : add validator's stake to threshold_stake.
604 :
605 : return threshold_stake >= FD_TOWER_THRESHOLD_PCT
606 : ```
607 :
608 : The threshold check simulates voting for the current slot to expire
609 : stale votes. This is to prevent validators that haven't voted in a
610 : long time from counting towards the threshold stake. */
611 :
612 : int
613 : fd_tower_threshold_check( fd_tower_t const * tower,
614 : fd_fork_t const * fork,
615 : fd_acc_mgr_t * acc_mgr );
616 :
617 : /* fd_tower_best_fork picks the best fork, where best is defined as the
618 : fork head containing the highest stake-weight in its ancestry.
619 : Returns a non-NULL fork. Assumes forks->frontier is non-empty. Note
620 : this is not necessarily the same fork as the one we vote on, as we
621 : might be locked out on a different fork.
622 :
623 : Does not modify tower. */
624 :
625 : fd_fork_t const *
626 : fd_tower_best_fork( fd_tower_t const * tower, fd_forks_t const * forks, fd_ghost_t const * ghost );
627 :
628 : /* fd_tower_reset_fork picks which fork to reset PoH to for our next
629 : leader slot. Returns a non-NULL fork. Note this is not necessarily
630 : the same fork as the one we vote on, as we do not always vote for the
631 : fork we reset to.
632 :
633 : Does not modify tower. */
634 :
635 : fd_fork_t const *
636 : fd_tower_reset_fork( fd_tower_t const * tower, fd_forks_t const * forks, fd_ghost_t const * ghost );
637 :
638 : /* fd_tower_vote_fork picks which frontier fork to vote on. Returns NULL
639 : if we cannot vote because we are locked out, do not meet switch
640 : threshold, or fail the threshold check.
641 :
642 : Modifies the tower to record the vote slot of the fork we select. */
643 :
644 : fd_fork_t const *
645 : fd_tower_vote_fork( fd_tower_t * tower,
646 : fd_forks_t const * forks,
647 : fd_acc_mgr_t * acc_mgr,
648 : fd_ghost_t const * ghost );
649 :
650 : /* fd_tower_epoch_update updates the tower after with a new epoch ctx.
651 : This should only be called on startup and when crossing an epoch
652 : boundary. */
653 :
654 : void
655 : fd_tower_epoch_update( fd_tower_t * tower, fd_exec_epoch_ctx_t const * epoch_ctx );
656 :
657 : /* fd_tower_fork_update updates ghost with the latest state of the vote
658 : accounts after executing the fork head (fork->slot).
659 :
660 : IMPORTANT SAFETY TIP! This should be called _after_ execution of
661 : fork->slot, not before. */
662 :
663 : void
664 : fd_tower_fork_update( fd_tower_t const * tower,
665 : fd_fork_t const * fork,
666 : fd_acc_mgr_t * acc_mgr,
667 : fd_blockstore_t * blockstore,
668 : fd_ghost_t * ghost );
669 :
670 : /* fd_tower_simulate_vote simulates a vote on the vote tower for slot,
671 : returning the new height (cnt) for all the votes that would have been
672 : popped. */
673 :
674 : ulong
675 : fd_tower_simulate_vote( fd_tower_t const * tower, ulong slot );
676 :
677 : /* fd_tower_vote votes for slot. Assumes caller has already performed
678 : all the tower checks to ensure this is a valid vote. */
679 :
680 : void
681 : fd_tower_vote( fd_tower_t const * tower, ulong slot );
682 :
683 : /* fd_tower_is_max_lockout returns 1 if the bottom vote of the tower has
684 : reached max lockout, 0 otherwise. Max lockout is equivalent to 1 <<
685 : FD_TOWER_VOTE_MAX (equivalently, confirmation count is
686 : FD_TOWER_VOTE_MAX). So if the tower is at height FD_TOWER_VOTE_MAX,
687 : then the bottom vote has reached max lockout. */
688 :
689 : static inline int
690 0 : fd_tower_is_max_lockout( fd_tower_t const * tower ) {
691 0 : return fd_tower_votes_cnt( tower->votes ) == FD_TOWER_VOTE_MAX;
692 0 : }
693 :
694 : /* fd_tower_publish publishes the tower. Returns the new root. Assumes
695 : caller has already checked that tower has reached max lockout (see
696 : fd_tower_is_max_lockout). */
697 :
698 : static inline ulong
699 0 : fd_tower_publish( fd_tower_t * tower ) {
700 0 : #if FD_TOWER_USE_HANDHOLDING
701 0 : FD_TEST( fd_tower_is_max_lockout( tower ) );
702 0 : #endif
703 :
704 0 : ulong root = fd_tower_votes_pop_head( tower->votes ).slot;
705 0 : FD_LOG_NOTICE( ( "[%s] root %lu", __func__, tower->root ) );
706 0 : tower->root = root;
707 0 : return root;
708 0 : }
709 :
710 : /* fd_tower_print pretty-prints tower as a formatted table.
711 :
712 : Sample output:
713 :
714 : slot | confirmation count
715 : --------- | ------------------
716 : 279803918 | 1
717 : 279803917 | 2
718 : 279803916 | 3
719 : 279803915 | 4
720 : 279803914 | 5
721 : 279803913 | 6
722 : 279803912 | 7
723 :
724 : */
725 :
726 : void
727 : fd_tower_print( fd_tower_t const * tower );
728 :
729 : /* Vote state API */
730 :
731 : /* fd_tower_vote_state_cmp compares tower with vote_state. Conceptually
732 : this is comparing our local view of our tower with the cluster's view
733 : (which may be out of sync). Returns -1 if the vote_state is
734 : newer, 0 if they're in sync and 1 if the local view is newer.
735 : Assumes both tower and vote_state are valid ie. there is a root and
736 : there is at least one vote (verifies this with handholding enabled).
737 :
738 : If tower is newer than vote_state, then the cluster has a stale view
739 : of our local tower. This normally just means our last vote hasn't
740 : landed yet and our vote state will eventually updated once that vote
741 : or a later one does land.
742 :
743 : If vote_state is newer than tower, then we already voted for
744 : fork->slot. This means we are not caught up yet or more
745 : problematically there is potentially another process running that is
746 : voting using our private key.
747 :
748 : IMPORTANT SAFETY TIP! Even though these towers may be out of sync,
749 : they should never be incompatible. For example, tower should never
750 : contain a state that could only be reached from vote_state by
751 : violating lockouts. */
752 :
753 : int
754 : fd_tower_vote_state_cmp( fd_tower_t const * tower, fd_vote_state_t * vote_state );
755 :
756 : /* fd_tower_vote_state_query queries for vote_acc_addr's vote state
757 : which is effectively the cluster view of the tower as of fork->slot.
758 : Returns a pointer to vote_state on success, NULL on failure. The
759 : vote_state is allocated using the provided valloc. */
760 :
761 : fd_vote_state_t *
762 : fd_tower_vote_state_query( fd_tower_t const * tower,
763 : fd_pubkey_t const * vote_acc_addr,
764 : fd_acc_mgr_t * acc_mgr,
765 : fd_fork_t const * fork,
766 : fd_valloc_t valloc,
767 : fd_vote_state_versioned_t * versioned );
768 :
769 : /* fd_tower_from_vote_state replaces the votes and root of tower with
770 : those from the vote state. */
771 :
772 : void
773 : fd_tower_from_vote_state( fd_tower_t * tower, fd_vote_state_t * vote_state );
774 :
775 : /* fd_tower_to_tower_sync converts an fd_tower_t into a fd_tower_sync_t
776 : to be sent out as a vote program ix inside a txn. */
777 :
778 : void
779 : fd_tower_to_tower_sync( fd_tower_t const * tower,
780 : fd_hash_t const * bank_hash,
781 : fd_compact_vote_state_update_t * tower_sync );
782 :
783 : #endif /* HEADER_fd_src_choreo_tower_fd_tower_h */
|
