diff a49999c..984e62e spherical_perceptron_bi_orthogonal_v2.txt
diff --git a/spherical_perceptron_bi_orthogonal_v2.txt b/spherical_perceptron_bi_orthogonal_v2.txt
index 97ac7d3..30e2e46 100644
--- a/spherical_perceptron_bi_orthogonal_v2.txt
+++ b/spherical_perceptron_bi_orthogonal_v2.txt
@@ -7,11 +7,11 @@ $$F_N(A,f)=\frac1N\log\int_{S^N}\exp\left\{\sum_{r=1}^M f((Ax)_r)\right\}d\sigma
The measurement matrix $A$ carries a disorder distribution on real
$M\times N$ matrices. A temperature parameter enters by writing
-$f=\beta\phi$. The present work determines the limit of $F_N(A,f)$ as
-$M/N\to\alpha>0$ for a class of disorder laws with bi-orthogonal
-invariance and prescribed compact limiting singular spectrum, and it
-produces an explicit variational formula in which both the activation
-and the entire limiting spectral distribution appear.
+$f=\beta\phi$. The limit of $F_N(A,f)$ as $M/N\to\alpha>0$ for a
+class of disorder laws with bi-orthogonal invariance and prescribed
+compact limiting singular spectrum is an explicit variational formula
+in which both the activation and the entire limiting spectral
+distribution appear.
The perceptron free energy has a long history. Gardner computed the
storage capacity of the spherical and binary perceptrons by a
@@ -22,27 +22,24 @@ the main statistical-mechanics picture and, in parallel, the
replica-symmetry-breaking predictions that later work tried to prove.
The Gardner phase diagram is the base against which the general
finite-temperature problem is compared.
-./refs/gardner-1988.pdf
-./refs/gardner-derrida-1988.pdf
For nonnegative-margin hard constraints, Shcherbina and Tirozzi
rigorously proved the replica-symmetric formula for the typical
spherical feasible volume. That result is the first complete
mathematical justification of the Gardner prediction in a spherical
-setting. The hard-margin problem is the zero-temperature special case
-of the pressure treated here, so the present result refines and
-enlarges the Shcherbina-Tirozzi theorem to finite temperature and to
-general spectrally supported disorder.
-./refs/shcherbina-tirozzi-2003.pdf
+setting. The hard-margin problem is the zero-temperature case of the
+finite-temperature pressure. The finite-temperature extension to
+general spectrally supported disorder refines and enlarges the
+Shcherbina-Tirozzi theorem.
The finite-temperature overlap-path formalism has substantial earlier
antecedents in physics. Gyorgyi and Reimann derive a nested Gaussian
recursion and a spherical entropy within a continuous
-replica-symmetry-breaking ansatz. Their formulation keeps a general
+replica-symmetry-breaking ansatz. The Gyorgyi-Reimann formulation keeps a general
scalar potential and then specialises to error counting beyond storage
capacity. Montanari and Zhou give a replica derivation of a
finite-temperature variational functional for general projection
-potentials with independent Gaussian patterns; their rigorous theorems
+potentials with independent Gaussian patterns; the rigorous theorems
concern the performance of approximate-message-passing algorithms.
These papers predict an overlap hierarchy beyond the regime described
by a single scalar order parameter. The variational formula stated
@@ -57,12 +54,11 @@ Shinzato and Kabashima derive the replica-symmetric expression and a
one-step replica-symmetry-breaking expression, and apply the
replica-symmetric calculation to spherical weights. Abbara, Aubin,
Krzakala and Zdeborova subsequently used finite-temperature free-energy
-calculations for rotationally invariant perceptrons in their study of
+calculations for rotationally invariant perceptrons in a study of
Rademacher complexity. These are direct predecessors of the spectral
-model treated here. The present theorem closes the loop: a limiting
-pressure is established through a full matrix-path variational problem
-for bounded continuous potentials with compact limiting singular
-spectrum and no outliers.
+model just described. A full matrix-path variational problem
+delivers a limiting pressure for bounded continuous potentials with
+compact limiting singular spectrum and no outliers.
./refs/kabashima-2008.pdf
./refs/arxiv-panchenko-vector.pdf
./refs/abbara-et-al-2020-pmlr.pdf
@@ -98,10 +94,10 @@ method of Mourrat, and its lower bound uses the cavity principle of
Aizenman, Sims and Starr. The hierarchical Gaussian fields driving the
upper bound sit inside the Hamilton-Jacobi framework developed by
Mourrat and by Mourrat and Panchenko. Those works provide the
-comparison framework; their theorems do not directly cover the present
-nonlinear pattern Hamiltonian. In the present setting the interaction
-parameter is the number of patterns and the field parameters are
-matrix covariance increments on the active Stiefel coordinates. The
+comparison framework; those theorems do not directly cover nonlinear
+pattern Hamiltonians. For the perceptron the interaction parameter is
+the number of patterns and the field parameters are matrix covariance
+increments on the active Stiefel coordinates. The
proof establishes the corresponding contact identities while
retaining the prescribed sharing of one coordinate.
./refs/arxiv-2004.01679v8.pdf
@@ -225,10 +221,10 @@ Define
$$\mathcal B_{p,a}(\kappa,\lambda,B,\theta)=\sup_{R_1,\ldots,R_h}\left[\sum_jp_j\left(\log\left(\kappa\sqrt{\frac{2\pi e}{p_j}}\right)+\log\frac{R_{j,11}}\kappa\right)+\theta\sum_{\ell=2}^{d_j}p_j\log\frac{R_{j,\ell\ell}}\kappa+\theta\min_{0\le t_\ell\le\lambda/\theta}\left\{t\cdot c+\mathcal G_{\theta,p}\left(\frac{f(y)}{\theta}-t\cdot g\right)\right\}\right],$$
where $\mathcal G_{\theta,p}$ is the Gaussian functional obtained from
-the powered Stiefel integral in the next paragraph. The first column of
+the powered Stiefel integral. The first column of
$R_j$ has weight $p_j$ and fires the outer entropy. The remaining
-columns have weight $p_j\theta$ and their entropy vanishes as
-$\theta\downarrow 0$. The min over $t$ is a Lagrangian dual for the
+columns have weight $p_j\theta$ and the entropy of those columns
+vanishes as $\theta\downarrow 0$. The min over $t$ is a Lagrangian dual for the
row-average constraints $M^{-1}\sum_r g_\ell(y_r,\dots)\le c_\ell$ in
the limit $\lambda\to\infty$. The truncation $B$ is removed last.
@@ -259,9 +255,9 @@ The first column is drawn once before the inner columns are integrated.
In the cascade representation it is shared by visits whose common level
is at least $\theta$; the other columns remain visit variables. The
matrix paths introduced next record the covariances of Gaussian row
-fields seen by such visits. In the Gaussian theorem they determine a
-variational functional whose value matches the limit of the normalized
-log integral.
+fields seen by such visits. In the Gaussian theorem those matrix paths
+determine a variational functional whose value matches the limit of the
+normalized log integral.
Choose a common finite grid
@@ -336,7 +332,7 @@ negative quadratic form is the shape produced when a shrinkage penalty
enforces bounded squared norm. A positive quadratic would increase
exponentially with the field and break the variational formula, so the
sign restriction excludes exactly that case. In $\mathcal B_{p,a}$ the
-row tests $g_\ell$ are nonnegative quadratics, and their contributions
+row tests $g_\ell$ are nonnegative quadratics, and the contributions
$-t\cdot g$ fall in the negative-quadratic class.
Reparametrise the powered integral as the partition function of a
@@ -364,7 +360,7 @@ function over independent patterns and independent disorder, standard
subadditivity and Gaussian concentration bound the variance of
$M^{-1}\log\mathcal Z_{M,\theta}(H)$ by $O(M^{-1})$. Consequently the
normalised log concentrates at its mean, and convergence in probability
-and convergence in expectation are equivalent for the present purpose.
+and convergence in expectation are equivalent.
Overlaps and matrix synchronization. Write replicas as superscripts
$a,b$. The scalar and matrix overlaps of a configuration $(X,A)$ at
@@ -379,7 +375,7 @@ dense set of vectors $v$, together with the numerical label overlap.
Each such scalarization is a Gram kernel: take tensor products of the
column vectors for the entrywise powers. Products, nonnegative
integral powers, and positive sums remain Gram kernels. After
-individual rescaling their absolute values are at most one and their
+individual rescaling the absolute values are at most one and the
diagonals are constant.
For a Hamiltonian with these joint scalarizations added and small
@@ -407,7 +403,7 @@ first-power forms $\alpha$ and antisymmetric perturbations $\beta$ of
$\mathcal R_j^{12}$ would generate crossed values on edges $12$ and
$13$ that the GG and ultrametricity structure forbid. Ultrametricity
of each of two crossed coordinates forces edge $23$ to have the lower
-value in each coordinate, contradicting ultrametricity of their sum.
+value in each coordinate, contradicting ultrametricity of the sum.
The required form of the ultrametric summary is the version proved by
Panchenko for vector spins and for the Potts spin glass, together
with the marked-cascade rules of Panchenko-Talagrand and Bolthausen-
@@ -580,7 +576,7 @@ patterns and $r=1$ recovers the true problem. Add independent
monomial perturbations of the pattern Hamiltonian scaled by
$s_r=r\sqrt M e_M$ with $e_M=M^{-1/16}$ and suitably normalised
coefficients. The perturbation kernels are those scalarizations used
-above; their Gaussian realizations are achieved by tensor fields on
+above; the Gaussian realizations are achieved by tensor fields on
the spin columns.
Define the contact functional
@@ -716,7 +712,7 @@ are drawn at visits. Restrict all normalised triangular lengths and
cross readings to a fixed small neighbourhood of the identity. The
first restriction and each conditional inner restriction have
probability tending to one independently of the preceding frame, and
-their recursion normalisation costs are $o(n)$.
+the recursion normalisation costs are $o(n)$.
The enlarged Gibbs measure with the $M$ original patterns and the
$\ell$ fresh ones has self covariance close to the identity and cross
@@ -751,7 +747,7 @@ $\ell$ via Markov. Perturbation replacement therefore preserves the
same prediction with the base overlap on the right side.
Taylor expansion on the shell. Expansion of the fresh pattern row
-fields around their base values gives a linear Gaussian field coupled
+fields around the base values gives a linear Gaussian field coupled
to each row of the fresh spin columns, of the form
$n_j^{-1/2}\sum_{r\le M}h_j^r(D'_j(\xi^r))^T$. The exact denominator
$\sqrt{n_j+r_j}$ may be replaced by $\sqrt{n_j}$ because $r_j$ is
@@ -767,7 +763,7 @@ $$W(X,Z)=\frac12\sum_jr_jc_j(X)+O(\epsilon\ell).$$
The first-order upper triangular part of $T_j-I$ has symmetric sum
$-Z_j^TZ_j/n_j$; at $Z_j^TZ_j=r_jI$ this is $-r_jI/(2n_j)$. The mean
fresh quadratic term is the Hessian contracted with
-$Z_j^TZ_j/(2n_j)$. Their diagonal specialisation is $r_jc_j(X)/2$,
+$Z_j^TZ_j/(2n_j)$. The diagonal specialisation is $r_jc_j(X)/2$,
and the retained matrix coefficients are bounded, so moving within
the shell costs $O(\epsilon r_j)$. Summing over $j$ gives the
correction. Cross-species fresh quadratic terms are centered.
@@ -776,7 +772,7 @@ Canonical prediction. Restricted to the thick shell, the Gaussian
predicts one specific matrix overlap path for the fresh-coordinate
pattern pair. Conditioning on the transformed unmarked sampled shape,
the canonical spatial rows are iid jointly Gaussian across replicas;
-their self covariance is $I$ and their cross covariance at common
+the self covariance is $I$ and the cross covariance at common
level $i$ is $q_{j,i}^*$ from the canonical lemma. The independent root
rows are averaged in this assertion, and additive row energies with
new-pattern energies factor the tilted kernels, so new patterns do not
@@ -786,9 +782,9 @@ The comparison. Physical rotation invariance predicts that the
sampled cross overlap approximates the empirical matrix overlap of the
enlarged system. Gaussian calculation on the same thick shell
predicts that the same sampled cross overlap approximates $q_{j,i}^*$.
-Both predictions hold at the sample coordinate level, and their
+Both predictions hold at the sample coordinate level, and the
failure probabilities bound the weighted path distance by twice the
-tolerance plus a bounded multiple of their sum. Therefore the
+tolerance plus a bounded multiple of the sum. Therefore the
empirical matrix overlap of the enlarged system is close to the
canonical path $q^*$.
@@ -894,8 +890,8 @@ $$\min_{t\in[0,d]^s}\{t\cdot c+\mathcal G_{\theta,p}(D-t\cdot g)\}.$$
The unpenalised functional $\mathcal G_{\theta,p}(D-t\cdot g)$ is
jointly convex in $t$ by direct differentiation and continuity of the
-unpenalised pressure in $h$ under the terminal class; we do not need a
-separate lower-semicontinuity hypothesis because the compact box and
+unpenalised pressure in $h$ under the terminal class; no separate
+lower-semicontinuity hypothesis is needed, because the compact box and
boundedness of $D$ on compacts ensure it. The minimum is attained, and
the value is finite and continuous in $(c,D,g)$ on compacts.
@@ -934,7 +930,7 @@ $$J_p=\frac12\int_0^p\log(1-u)\,du.$$
Indeed the successive orthogonal residual lengths of $G$ are
independent with laws $(\chi^2_{M-r+1}/M)^{1/2}$, chi-square
-concentration bounds their normalised logarithms, and a Riemann sum
+concentration bounds the normalised logarithms, and a Riemann sum
gives the integral.
Coefficient integral. Write $A=U\mathrm{diag}(s)V^T$ and put
@@ -962,7 +958,7 @@ prescribed total mass. After sorting $a_j/p_j$, the weighted shell
lemma shows that these one-sided tests recover the desired weighted
shell of radii $a_j$. The witnesses are integrated as the inner
variables of the powered Stiefel integral, with exponent $\theta\to 0$
-removing their entropy.
+removing the entropy.
Coefficient model. With bounded $f$, the coefficient integral over
$S^n$ decomposes into $\int\prod_jd\mu_j$ where $\mu_j$ is uniform on
@@ -999,7 +995,6 @@ The allocation step is the finite-weight Hardy-Littlewood-Polya
convex-order principle. For the original rearrangement background see
Hardy, Littlewood, and Polya's monograph; the specific weight-preserving
allocation proof is given here.
-./refs/hlp-1952-inequalities.pdf
Consider allocation tables $\lambda_{ij}\ge 0$ with column sums $p_j$
and row sums $p_i$. The entry $\lambda_{ij}$ is the dimension
@@ -1023,13 +1018,13 @@ The relation is majorization relative to the block proportions $p$;
see the class of $d$-majorization polytopes studied by vom Ende and
Dirr. Realise it geometrically by randomly subdividing each old $E_j$
into orthogonal pieces of relative dimensions approaching $T_{ij}$,
-and forming new block $i$ as their direct sum. The identities $Tp=p$
+and forming new block $i$ as the direct sum. The identities $Tp=p$
and $Tb=a$ give respectively the new dimension proportions and the
target squared masses. Dimensions with the correct two margins are
obtained by $o(M)$ adjustments and bounded rounding corrections. The
new decomposition has the same Haar marginal as the old one. For each
fixed vector in the old $b$-shell, squared norms in the pieces
-concentrate about their prescribed fractions; the failure bound is
+concentrate about the prescribed fractions; the failure bound is
uniform in the fixed vector. Simultaneous concentration for all shell
vectors is unnecessary. Thus each such vector lies in a slightly
enlarged new $a$-shell with failure probability $\epsilon_M\to 0$.
@@ -1057,7 +1052,7 @@ The concentration rate for the coefficient integral at that error is
independent of $B,\theta$. Choose $\eta$ so that the entropy
$\frac12\log\eta$ is below half this rate and so that
$2\|f\|_\infty\eta+\frac12\log\eta<\epsilon$. Choose shell widths and
-$\delta$ so their errors times $\lambda$ are small, then choose a
+$\delta$ so the errors times $\lambda$ are small, then choose a
sufficiently large $B$, independently of $\theta$. Finally take
$\theta$ small enough that the witness entropy is below $\epsilon$. A
union bound over the candidate rotations and the concentration
@@ -1117,15 +1112,15 @@ $$x=\sqrt{\rho_0N}\theta_0+\sum_j\sqrt{\rho_jN/s_j^2}\theta_j\sqrt{s_j^2/N},$$
where $\rho_j\ge 0$ is the fraction of squared norm in block $j$ (with a
null block of size $N-M$ when $\alpha<1$), $\sum_j\rho_j=1$, and the
-$\theta_j$ are unit vectors in their blocks. For a uniform coefficient
+$\theta_j$ are unit vectors in the blocks. For a uniform coefficient
$x$ the joint law factors: the $\theta_j$ are independent uniform unit
-vectors in their blocks, and $\rho=(\rho_j)$ is Dirichlet with
+vectors in the blocks, and $\rho=(\rho_j)$ is Dirichlet with
parameters $(n_j/2)$ (including the null block when it is present).
Under this split, the measurement is
$Ax=\sum_jU_j(s_j\sqrt{N\rho_j/\alpha})\,G_j\theta_j$ plus a null
contribution. Block $j$ has image subspace $E_j=U_j\mathbb R^{n_j}$ of
-haar dimension $n_j$ in $\mathbb R^M$, and the vector
+Haar dimension $n_j$ in $\mathbb R^M$, and the vector
$y_j=s_j\sqrt{N\rho_j}G_j\theta_j$ is uniform on the sphere of radius
$\sqrt{Ma_j}$ in $E_j$ with $a_j=s_j^2\rho_j/\alpha$. The $y_j$ are
mutually independent once $\rho$ is fixed.
@@ -1190,7 +1185,7 @@ with $v=\|x\|^2/N\sim\mathrm{Beta}(M/2,k/2)$ and $x'$ uniform on
$S^{M-1}$ conditionally. The measured field at radius $\sqrt{Nv}$ has
squared norm approximately $\alpha s_N^2v$ when $s_N$ is the typical
singular value. The endpoints $v$ small and $v$ near one contribute
-negligibly because the Beta density makes them exponentially rare at
+negligibly because the Beta density makes such values exponentially rare at
the field-energy scale. Integration by parts over $v$ converts the
sphere integral into a ball integral of $Z_M(r)$ over
$r^2\in[\alpha s_-^2v_{\rm lo},\alpha s_+^2v_{\rm hi}]$ times a
@@ -1209,7 +1204,7 @@ $$\max_{r\le\min(M,N)}\mathrm{dist}(s_{N,r},[s_-,s_+])\to 0\quad\hbox{in probabi
The edge interval suffices. If a sublinear number of singular values
leak outside $[s_-,s_+]$, one or two extra Haar directions act as
-fake blocks. The leaked weight is assigned to them, and the negative
+fake blocks. The leaked weight is assigned to those directions, and the negative
Dirichlet entropy $\frac12v_i\log(\rho_i/v_i)$ with $v_i=o(1)$
absorbs the effect as $\rho_i\to 0$. When $\alpha<1$, the interval may
be $[0,s_+]$, because the macroscopic input null space provides a
@@ -1267,8 +1262,7 @@ $\{z\ne 0\}$ can oscillate between two values along a sequence whose
rectangular singular slots converge weakly with no outlier in the
support of the limiting measure. Sublinear outliers that hide behind
structural output zeros break the edge interval hypothesis when the
-zero endpoint is not a legitimate input null. The next paragraph
-gives four worked counterexamples.
+zero endpoint is not a legitimate input null. Four worked counterexamples follow.
Counterexample: forced output zeros hide a small singular outlier.
Let $M=1000N$ so $\alpha=1000$. On even $N$ take $A_N$ equal to