Documentation

DescriptiveComplexity.FixedPoint

FO(LFP): first-order logic with a least fixed point #

The logic FO(LFP) (Immerman 1986; Vardi 1982), in the clausal normal form this library uses for kernels everywhere else: a DescriptiveComplexity.LFPDef bundles

Why the output formula is the point #

The fixed point is the same object in both logics – the least model of a set of rules. What distinguishes FO(LFP) from SO-Horn is what one is allowed to say about it. An SO-Horn program can only reject models, through goal clauses, and a goal clause tests its atoms positively; FO(LFP) evaluates an unrestricted first-order sentence, so it may negate fixed-point atoms.

That is exactly the closure that SO-Horn lacks: DescriptiveComplexity.LFPDefinable is closed under complement by negating the output (DescriptiveComplexity.LFPDefinable.compl, a one-liner), whereas the corresponding statement for SO-Horn is open – see DescriptiveComplexity.Problems.HornSat. The inclusion DescriptiveComplexity.SigmaSOHornDefinable.lfpDefinable transports every SO-Horn definition into this logic, so PTIME as defined by the Horn fragment sits inside FO(LFP) together with its complements.

What is proved here, and where the equivalence is completed #

DescriptiveComplexity.SigmaSOHornDefinable.lfpDefinable is one half of the equivalence of the two formalisms: every SO-Horn definition is an FO(LFP) definition. The other half – bringing an FO(LFP) definition back into the Horn fragment – is the translation of DescriptiveComplexity.FixedPointHorn, built on the stage theory of this file: the stages DescriptiveComplexity.derivesIn stabilize once the atom count is reached (DescriptiveComplexity.derivesIn_iff_derives_of_card_le), so a stage indexed by a large enough tuple stands in for the fixed point, and its complement can be derived positively, one stage at a time. Together the two halves make the notions interchangeable (DescriptiveComplexity.lfpDefinable_iff_sigmaSOHornDefinable) and give PiP 0 = SigmaP 0 (DescriptiveComplexity.piP_zero_eq).

The notion is also closed under (ordered) first-order reductions (DescriptiveComplexity.LFPDefinable.of_orderedReduction), so it is class-worthy in the sense of DescriptiveComplexity.ComplexityClass.

A second consumer of the stage theory is not formalized: a Σ₁ definition, giving FO(LFP) ⊆ NP directly, would guess a fixed point together with a well-founded derivation order and check both first-order. The semantic key is provided here – DescriptiveComplexity.derives_eq_of_closed_of_wf says that a relation closed under the rules and well-foundedly derivable is exactly the least fixed point, DescriptiveComplexity.derives_step_of_depth supplies the witnessing order from the stages, and DescriptiveComplexity.LFPDef.holds_iff_of_certificate packages this – but the formula building remains to be done. (The inclusion itself already follows by composing the translation with the Horn membership of DescriptiveComplexity.Problems.HornSat.)

The fixed point #

Rather than a stage-indexed iteration, the least model is the inductive predicate DescriptiveComplexity.Derives: a tuple is derived when some rule fires on already-derived tuples. Being an inductive definition it comes with exactly the two properties a least fixed point needs – it satisfies the rules (DescriptiveComplexity.lfpAssign_rule) and it is contained in every assignment that does (DescriptiveComplexity.lfpAssign_least).

Rules whose head is none derive nothing and are simply inert here, so an SO-Horn program can be handed over unchanged: its goal clauses reappear in the output formula.

The least model of a rule system #

inductive DescriptiveComplexity.Derives {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] (rules : List (HornClause Lg B k)) :
(i : B.ι) × (Fin (B.arity i)A)Prop

The tuples derivable by a system of rules: the least fixed point, as an inductive predicate. A rule fires when its guard holds and all its body atoms are already derived, and it derives its head atom.

Instances For
    Dependency graph
    def DescriptiveComplexity.lfpAssign {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] (rules : List (HornClause Lg B k)) :

    The least fixed point of a rule system, as an assignment of the block.

    Equations
    Instances For
      Dependency graph
      theorem DescriptiveComplexity.lfpAssign_rule {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {c : HornClause Lg B k} (hc : c rules) {a : SOAtom B k} (ha : c.head = some a) (v : Fin kA) :
      c.Holds (lfpAssign rules) v

      The least fixed point satisfies every rule with a head. (A clause with no head is a constraint, not a rule: it derives nothing, and is what the output formula of DescriptiveComplexity.LFPDef takes over.)

      Dependency graph
      theorem DescriptiveComplexity.lfpAssign_least_of_closed {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {ρ : B.Assignment A} ( : crules, ∀ (a : SOAtom B k), c.head = some a∀ (v : Fin kA), c.guard.Realize v(∀ bc.body, b.Holds ρ v)a.Holds ρ v) {p : (i : B.ι) × (Fin (B.arity i)A)} (hp : Derives rules p) :
      ρ p.fst p.snd

      The least fixed point is contained in every prefixpoint: in every assignment closed under the rules.

      Dependency graph
      theorem DescriptiveComplexity.lfpAssign_least {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {ρ : B.Assignment A} ( : ∀ (v : Fin kA), crules, c.Holds ρ v) {p : (i : B.ι) × (Fin (B.arity i)A)} (hp : Derives rules p) :
      ρ p.fst p.snd

      The least fixed point is contained in every assignment satisfying the rules.

      Dependency graph

      Stages, depth, and the certificate characterization #

      Putting FO(LFP) back into the Horn fragment, and certifying a fixed point in Σ₁, both need the same thing: a way to say “this relation is the least fixed point” that a positive formalism can test. Closure alone is not enough (anything larger is closed too); what pins the least fixed point down is closure together with well-founded derivability – every element derived by a rule from strictly earlier elements. That is DescriptiveComplexity.derives_eq_of_closed_of_wf below, and the stages provide the witnessing order.

      @[reducible, inline]

      The atoms of a block over a structure: a relation variable and a tuple.

      Equations
      Instances For
        Dependency graph
        def DescriptiveComplexity.stepDerives {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] (rules : List (HornClause Lg B k)) (S : BAtom B AProp) :
        BAtom B AProp

        One application of the rules to a set of atoms.

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          Dependency graph
          def DescriptiveComplexity.derivesIn {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] (rules : List (HornClause Lg B k)) :
          BAtom B AProp

          The stages of the derivation: what is derivable in at most n rounds.

          Equations
          Instances For
            Dependency graph
            theorem DescriptiveComplexity.derivesIn_succ {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {n : } {q : BAtom B A} (h : derivesIn rules n q) :
            derivesIn rules (n + 1) q
            Dependency graph
            theorem DescriptiveComplexity.derivesIn_le {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {m n : } (hmn : m n) {q : BAtom B A} (h : derivesIn rules m q) :
            derivesIn rules n q
            Dependency graph
            theorem DescriptiveComplexity.derives_iff_derivesIn {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {q : BAtom B A} :
            Derives rules q ∃ (n : ), derivesIn rules n q

            The stages exhaust the least fixed point.

            Dependency graph

            Depth, and the certificate #

            noncomputable def DescriptiveComplexity.depth {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] (rules : List (HornClause Lg B k)) (q : BAtom B A) :

            The stage at which an atom first appears.

            Equations
            Instances For
              Dependency graph
              theorem DescriptiveComplexity.derivesIn_depth {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {q : BAtom B A} (h : Derives rules q) :
              derivesIn rules (depth rules q) q
              Dependency graph
              theorem DescriptiveComplexity.depth_le {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {n : } {q : BAtom B A} (h : derivesIn rules n q) :
              depth rules q n
              Dependency graph
              theorem DescriptiveComplexity.derives_step_of_depth {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {q : BAtom B A} (h : Derives rules q) :
              crules, ∃ (a : SOAtom B k), c.head = some a ∃ (v : Fin kA), q = a.idx, fun (j : Fin (B.arity a.idx)) => v (a.args j) c.guard.Realize v bc.body, Derives rules b.idx, fun (j : Fin (B.arity b.idx)) => v (b.args j) depth rules b.idx, fun (j : Fin (B.arity b.idx)) => v (b.args j) < depth rules q

              Every derived atom is derived from atoms of strictly smaller depth: the least fixed point comes with a well-founded derivation order.

              Dependency graph

              Stabilization: the stages close within Nat.card many rounds #

              On a finite structure the stages are an increasing chain of subsets of the finitely many atoms, so they stabilize by the time the atom count is reached – after that many rounds, DescriptiveComplexity.derivesIn is the least fixed point. This is what lets a stage indexed by a large enough tuple stand in for the fixed point itself in DescriptiveComplexity.FixedPointHorn.

              theorem DescriptiveComplexity.derivesIn_iff_derives_of_card_le {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] [Finite A] {rules : List (HornClause Lg B k)} {r : } (hr : Nat.card (BAtom B A) r) {q : BAtom B A} :
              derivesIn rules r q Derives rules q

              The stages stabilize at the atom count: after Nat.card (BAtom B A) many rounds, being derivable within that many rounds is being derivable.

              Dependency graph
              theorem DescriptiveComplexity.derives_eq_of_closed_of_wf {Lg : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [Lg.Structure A] {rules : List (HornClause Lg B k)} {R : BAtom B AProp} {lt : BAtom B ABAtom B AProp} (hwf : WellFounded lt) (hclosed : crules, ∀ (a : SOAtom B k), c.head = some a∀ (v : Fin kA), c.guard.Realize v(∀ bc.body, R b.idx, fun (j : Fin (B.arity b.idx)) => v (b.args j))R a.idx, fun (j : Fin (B.arity a.idx)) => v (a.args j)) (hderiv : ∀ (q : BAtom B A), R qcrules, ∃ (a : SOAtom B k), c.head = some a ∃ (v : Fin kA), q = a.idx, fun (j : Fin (B.arity a.idx)) => v (a.args j) c.guard.Realize v bc.body, R b.idx, fun (j : Fin (B.arity b.idx)) => v (b.args j) lt b.idx, fun (j : Fin (B.arity b.idx)) => v (b.args j) q) (q : BAtom B A) :
              R q Derives rules q

              The certificate characterization: a relation closed under the rules and all of whose atoms are derived from strictly -earlier ones, for a well-founded , is the least fixed point.

              This is what makes the least fixed point testable by a positive formalism: it is the shape a Σ₁ certificate for FO(LFP) would check. (The translation back into the Horn fragment, DescriptiveComplexity.FixedPointHorn, instead walks the stages directly, through their stabilization DescriptiveComplexity.derivesIn_iff_derives_of_card_le below.)

              Dependency graph

              The fixed point transports along isomorphisms #

              theorem DescriptiveComplexity.derives_map {Lg : FirstOrder.Language} {B : SOBlock} {k : } {M N : Type} [Lg.Structure M] [Lg.Structure N] (e : Lg.Equiv M N) {rules : List (HornClause Lg B k)} {q : BAtom B M} (h : Derives rules q) :
              Derives rules q.fst, fun (j : Fin (B.arity q.fst)) => e (q.snd j)

              Derivability transports along an isomorphism of structures: the rules only see the guards, which an isomorphism preserves.

              Dependency graph
              theorem DescriptiveComplexity.lfpAssign_map {Lg : FirstOrder.Language} {B : SOBlock} {k : } {M N : Type} [Lg.Structure M] [Lg.Structure N] (e : Lg.Equiv M N) (rules : List (HornClause Lg B k)) :

              The least fixed point transports along an isomorphism.

              Dependency graph

              The fixed point commutes with pullbacks #

              The least fixed point of the pulled rules is the pullback of the least fixed point: both inclusions are an application of leastness, since each side is closed under the other's rules (DescriptiveComplexity.HornProgram.pull_holds). This is the fixed-point half of closure of DescriptiveComplexity.LFPDefinable under (ordered) first-order reductions; what remains for that closure is to pull the output sentence back, through DescriptiveComplexity.FOInterpretation.extendSO and DescriptiveComplexity.FOInterpretation.ordExtendLEquiv, which is bookkeeping between the three structures involved rather than mathematics.

              theorem DescriptiveComplexity.lfpAssign_pull {L₁ L₂ : FirstOrder.Language} [L₂.IsRelational] {Tag : Type} [Finite Tag] {d : } {B : SOBlock} {k : } {A : Type} [L₁.Structure A] (I : FOInterpretation L₁ L₂ Tag d) (rules : List (HornClause L₂ B k)) :
              Dependency graph

              Definitions in FO(LFP) #

              A definition in FO(LFP), in clausal normal form: a block of relation variables, a list of rules defining them inductively, and a first-order output sentence over the vocabulary expanded by the variables, read at the least fixed point.

              Instances For
                Dependency graph

                The value of a definition on a structure: the output read at the least fixed point of the rules.

                Equations
                Instances For
                  Dependency graph

                  Negating the output complements the defined property: FO(LFP) is closed under complement by construction, being a logic rather than a fragment.

                  Equations
                  Instances For
                    Dependency graph
                    Dependency graph
                    theorem DescriptiveComplexity.LFPDef.holds_iff_of_certificate {L : FirstOrder.Language} (d : LFPDef L) {A : Type} [L.Structure A] [LinearOrder A] {ρ : d.B.Assignment A} {lt : BAtom d.B ABAtom d.B AProp} (hwf : WellFounded lt) (hclosed : cd.rules, ∀ (a : SOAtom d.B d.k), c.head = some a∀ (v : Fin d.kA), c.guard.Realize v(∀ bc.body, b.Holds ρ v)a.Holds ρ v) (hderiv : ∀ (q : BAtom d.B A), ρ q.fst q.sndcd.rules, ∃ (a : SOAtom d.B d.k), c.head = some a ∃ (v : Fin d.kA), q = a.idx, fun (j : Fin (d.B.arity a.idx)) => v (a.args j) c.guard.Realize v bc.body, (ρ b.idx fun (j : Fin (d.B.arity b.idx)) => v (b.args j)) lt b.idx, fun (j : Fin (d.B.arity b.idx)) => v (b.args j) q) :
                    d.Holds A A d.out

                    A certified fixed point can be used in place of the real one. If ρ is closed under the rules and every one of its atoms is derived from -earlier ones, for a well-founded , then reading the output at ρ gives the value of the definition.

                    This is the interface a Σ₁ definition of an FO(LFP) definable problem would consume: guess ρ and and check these conditions first-order, coming from the stages (DescriptiveComplexity.derives_step_of_depth). The translation into the Horn fragment (DescriptiveComplexity.FixedPointHorn) instead derives the stages themselves.

                    Dependency graph

                    A decision problem is FO(LFP) definable if, on nonempty finite ordered structures, it is the value of a definition in FO(LFP). As for DescriptiveComplexity.SigmaSOHornDefinable, the equivalence is required for every linear order, so the notion is order-invariant.

                    Equations
                    Instances For
                      Dependency graph

                      FO(LFP) definability is closed under complement. This is the one line that the Horn fragment cannot supply, and the reason to have the logic at all: negate the output formula.

                      Dependency graph

                      SO-Horn definitions are FO(LFP) definitions #

                      A Horn program splits into its rules (the clauses with a head), which define the fixed point, and its goal clauses (those without), which merely say that the fixed point avoids certain configurations – a first-order statement about it, and so exactly what an output formula can express.

                      @[reducible, inline]

                      The symbol of a relation variable of the block.

                      Equations
                      Instances For
                        Dependency graph
                        @[reducible, inline]

                        The symbol of a relation variable, in the expanded vocabulary.

                        Equations
                        Instances For
                          Dependency graph

                          A second-order atom, as a first-order atom over the expanded vocabulary.

                          Equations
                          Instances For
                            Dependency graph
                            Dependency graph

                            A goal clause, as the first-order statement that it never fires.

                            Equations
                            • One or more equations did not get rendered due to their size.
                            Instances For
                              Dependency graph

                              The output formula of the translation: no goal clause of the program ever fires at the fixed point.

                              Equations
                              • One or more equations did not get rendered due to their size.
                              Instances For
                                Dependency graph
                                theorem DescriptiveComplexity.realize_atomF {L : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [L.Structure A] [LinearOrder A] (ρ : B.Assignment A) (a : SOAtom B k) (v : Empty Fin kA) :
                                (atomF a).Realize v a.Holds ρ fun (j : Fin k) => v (Sum.inr j)
                                Dependency graph
                                theorem DescriptiveComplexity.realize_guardOutF {L : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [L.Structure A] [LinearOrder A] (ρ : B.Assignment A) (φ : (L.sum FirstOrder.Language.order).Formula (Fin k)) (v : Empty Fin kA) :
                                (guardOutF φ).Realize v φ.Realize fun (j : Fin k) => v (Sum.inr j)
                                Dependency graph
                                theorem DescriptiveComplexity.realize_goalOutF {L : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [L.Structure A] [LinearOrder A] (ρ : B.Assignment A) (c : HornClause (L.sum FirstOrder.Language.order) B k) :
                                A goalOutF c ∀ (v : Fin kA), ¬(c.guard.Realize v bc.body, b.Holds ρ v)
                                Dependency graph
                                theorem DescriptiveComplexity.realize_hornOutF {L : FirstOrder.Language} {B : SOBlock} {k : } {A : Type} [L.Structure A] [LinearOrder A] (ρ : B.Assignment A) (prog : HornProgram (L.sum FirstOrder.Language.order) B k) :
                                A hornOutF prog cprog, c.head = none∀ (v : Fin kA), ¬(c.guard.Realize v bc.body, b.Holds ρ v)
                                Dependency graph

                                Every SO-Horn definition is an FO(LFP) definition: keep the rules, and turn the goal clauses into the output formula.

                                Dependency graph

                                The complements of the SO-Horn definable problems are FO(LFP) definable too – which is what the Horn fragment on its own cannot say.

                                Dependency graph

                                Closure under reductions #

                                FO(LFP) definability is closed under ordered first-order reductions. The rules pull back as a Horn program and their fixed point commutes with the pullback (DescriptiveComplexity.lfpAssign_pull); the output sentence pulls back through DescriptiveComplexity.FOInterpretation.extendSO, the two views of the interpreted structure being identified by DescriptiveComplexity.FOInterpretation.extendSOEquiv and DescriptiveComplexity.FOInterpretation.ordExtendLEquiv.

                                Dependency graph