31 we determined 3D electron density maps using single-particle cryo EM analysis 32 for three single-chain de-novo proteins comprising 600 AA which closely 33 matched with the designed shape. These have no structural analogues in the 34 protein data bank (PDB), representing potentially novel folds or arrangement of 35 domains. Our approach broadens the scope of de novo protein design and 36 contributes to accessibility to a wider community.

Deep-Learning (DL) based protein structure prediction methods such as AlphaFold2 (AF2)1 and RoseTTAFold2 can generalize beyond the sequence and structure space they have been trained upon to correctly predict the structures of de novo designed proteins3–6. Additionally, AlphaFold2 can distinguish between favorable and unfavorable structure templates7, suggesting that it has learned the physical properties of energetically stable protein backbones. DL-based structure prediction methods can thus connect sequence space and protein structure space, and enable searching for protein structures that satisfy a given design target. A DL based design method called ‘deep network hallucination’8 leverages this connection for a variety of protein design problems by iteratively updating a protein sequence until a desired property encoded in a mathematical loss function is obtained. Typically, when performing deep network hallucination, random mutations are applied in a Monte Carlo Markov Chain (MCMC) fashion4,6,9–11. However, this method can be computationally inefficient due to the need for a large number of iterations. This approach may also fail to identify a solution, similar to issues faced by numerical solutions to mathematical problems in a random Monte Carlo fashion. To improve the likelihood of generating accurate predictions, researchers have proposed using sequence gradients obtained by inverting structure prediction networks12. However, when updating the gradients, a major issue arises due to the discrete, one-hot encoded representation of sequences. One-hot encoding (Fig.: 1, A) represents each amino acid or nucleotide as a binary vector with all zeros, except for a single ‘1’, indicating the position of the corresponding residue. While the obtained gradient from the backpropagation through the structure prediction network is non discrete, previous approaches tried to regain discreteness by a ‘straight-through estimator’13 62 in the sequence update loop5,12,14. Though this approach worked well for models 63 predicting distribution of distances for every pair of positions, such as TrRosetta, the 64 approach did not work well in more recent models such as AlphaFold, where single 65 amino acid changes could radically alter the predicted structure, resulting in unstable 66 and inefficient optimization. Therefore, alternative methods of representing 67 sequences in a continuous format are being explored to overcome this problem. 68 69 The AF2 network can predict sequences generated through the hallucination process 70 with high confidence. However, despite this high confidence, the experimental 71 success rate of the resulting proteins is often low, resulting in the production of 72 insoluble proteins6,15. To resolve this issue, it has become standard practice to 73 generate a new sequence for the given backbone using the ProteinMPNN sequence 74 design software15. ProteinMPNN has demonstrated significant success in designing 75 soluble sequences for specific protein backbones, making it a valuable tool for 76 improving the experimental success rate of the resulting proteins. Since the sequence 77 design process for a given backbone geometry can be reliably handled with 78 ProteinMPNN, we hypothesized that using a continuous (relaxed) sequence 79 representation would allow for a more targeted and efficient gradient-descent search 80 through the structure space for desired backbone properties while avoiding the 81 issues associated with the need for a discrete, one-hot encoded sequence. A relaxed 82 sequence representation approach may thus allow for more precise and effective 83 exploration of the structure space for backbones, resulting in improved outcomes for 84 protein design. 85 86 To test our hypothesis, we utilized a representation where each amino acid position 87 in the input sequence is a vector of 20 values representing the contributions from 88 each of the 20 natural amino acids. These values can be fractional, greater than one, 89 or negative and positive in value (Fig.: 1, B). Though the input sequence, termed 90 "logits", may represent an unnormalized probability distribution of amino acids, in this 91 work they serve as a means to more efficiently explore the AF2 network for backbone 92 geometries. Specifically, the relaxed sequence representation enables adopting a 93 gradient-descent based hallucination protocol that optimizes a loss function with 94 rapid convergence towards high confidence protein structures (Fig.: 1, C). To 95 implement this relaxed sequence representation, we used the ColabDesign 96 framework, which offers various deep network hallucination protocols with a 97 modified, differentiable version of AF2. 98 99 Using this approach for backbone generation, we set up a protein design pipeline 100 that uses relaxed sequence hallucination to generate target backbone geometries. 101 After obtaining high-confidence backbone coordinates with a pLDDT score greater 102 than 90, we then use the ProteinMPNN network to generate actual physical candidate 103 sequences represented as a one-hot encoding (Fig.: 1, D). We then repredict the 104 candidate sequences by AlphaFold2 and ESM-fold16 to validate the one-hot 105 candidate sequences obtained from ProteinMPNN with respect to forming the input 106 backbone geometry. This step is necessary since AF2 models P(structure | sequence) 107 whereas MPNN models P(sequence | structure). It is thus possible that P(sequence | 108 wrong_structure) >= P(sequence | desired_structure). Therefore one needs to confirm 109 that P(desired_structure | sequence) is valid using a new pass through a structure 110 prediction model.

To test the efficiency of our relaxed sequence hallucination, we performed a benchmarking experiment in which we unconditionally hallucinated backbones with three different protocols: gradient-descent with relaxed sequence (GD relaxed); GD with argmax() and one-hot encoded sequences (GD hard); and Markov-Chain-MonteCarlo search (MCMC) within the ColabDesign framework (Fig.: 2, A). GD relaxed robustly outperformed the other design approaches in all cases tested: the average convergence “half-life” was 20 iterations across all design test cases, whereas GD hard and MCMC failed to converge to satisfactory results (pLDDT >90) within the tested number of sequence update steps (Fig.: 2, A & Supp. Fig.: 1, A). Notably the GD relaxed method reliably produced high confidence backbones with pLDDT score larger than 95 for all unconditional hallucination trajectories tested (Fig.: 2, A). We note GD relaxed utilizes a back propagation step to implement the gradient descent, as opposed to only performing forward passes through AF2 as in MCMC. Hence, the computational effort for one sequence update step is greater in GD relaxed, but because of the orders-of-magnitude reduction in steps needed to convergence, relaxed sequence hallucination strongly outperforms MCMC in terms of design speed (Supp. Fig.: 1, B).

The output of GD relaxed backbone hallucination is structurally diverse, including all helical to all beta and combinations of helical & sheet mixtures (Fig.: 2, B) and can also be utilized to hallucinate backbones with multiple subunits. To further benchmark our hallucination approach, we examined the backbone recovery17 over multiple sizes of proteins ranging from 100 to 1000 amino acids (AAs). To this end we generated eight candidate sequences for each GD relaxed generated target backbone using ProteinMPNN, and then re-predicted these candidate sequences with AlphaFold2 in single sequence mode. We calculated the root-mean-square-deviation and TMScore18 of the predicted structure relative to the initial backbone input and chose the best hit. We observed high agreement of predictions of MPNN-redesigned sequences to the target backbones over all protein sizes tested (Fig.: 2, C). Judging by the obtained median RMSD relative to the target backbone, relaxed sequence hallucination delivered median sub 4 angstrom backbone recovery for target proteins up to 600 AAs in size. The backbone recovery efficiency drops for backbones larger than 600 AAs (Fig.: 2, C).

To address the potential concern that the generated backbones are adversarial and specific to AlphaFold, we used an independently trained model ESM-fold, which was recently used to generate experimentally validated de novo designs9, as an orthogonal test. We see ESM-Fold is able to recapitulate the ProteinMPNN redesigned backbones in a similar fashion than AF2 with single sequence input (Fig.: 2, D).

To test whether our GD relaxed hallucination protocol could produce real-world expressible proteins we generated a test panel of monomeric protein backbones covering sizes ranging from 100 to 600 amino acids. The geometries were chosen to resemble a mixture of alpha-helical and beta-sheet structural elements. We then fed the resulting backbones to ProteinMPNN for candidate sequence generation and performed AF2 single sequence prediction verification. The candidate sequences were gene-synthesized, expressed in Escherichia coli (BL21), and purified through immobilized metal affinity chromatography (IMAC). 13 out of 14 tested protein designs, no matter their size, expressed well, were soluble and had the correct molecular weight as determined by SDS page analysis (Supp. Fig.: 2, A). Size exclusion chromatography elution profiles obtained under native conditions (Fig.: 3 (SEC)) were consistent with the expected sizes of the tested proteins and predominantly showed one single elution peak. All of the SEC tested proteins produced circular dichroism (CD) spectra that indicated that the proteins are wellfolded and featured the secondary structure elements that were expected based on their design. All of the tested proteins were also markedly thermostable, as indicated by the CD spectra remaining largely unaltered up to temperatures of 95°C (Fig.: 3, CD). We attribute these satisfactory results to favorable backbone and sequence generation by GD relaxed and ProteinMPNN working in concert and the AF2 single sequence filter. Indeed, one-hot-encoding the relaxed sequence representation without passing backbones through ProteinMPNN, resulted in largely insoluble proteins in our hands (Supp. Fig.: 2, B), consistent with previous findings9. With a pass through ProteinMPNN, the designed sequences expressed and folded according to expectation (Supp. Fig.: 2, B), in support of previous findings6,15. This shows the high value of ProteinMPNN in the design pipeline. Complementarily, the GD relaxed approach now efficiently generates high quality backbones.

We adapted our pipeline to design multimeric proteins (See Materials & Methods) and tested homo-dimeric and -trimeric candidate designs. All oligomeric designs produced soluble proteins with the expected size of the monomeric chain as determined by SDS-PAGE (Supp. Fig.: 3, A). The oligomeric proteins also had CD spectra that were in good agreement with the expected secondary structure (Supp. Fig.: 3, B). We used FPLC-SEC to evaluate whether the proteins actually formed oligomers as designed. This analysis indicated that out of our design candidates, two homodimers and two homotrimers formed (Fig.: 4, A & B). Next, we modified our pipeline to produce heterodimeric proteins. To this end, we installed a gap of 50 amino acids19 in the relaxed sequence input to AF2, which modifies the relative positional encoding to the maximum constant, since the relative encoding is clipped at sequence separation of 32. We hallucinated 1000 backbones and generated three candidate sequences for each backbone design with ProteinMPNN. We then used AF2 single sequence prediction to compute separately the structures of the two single chains in each heterodimers. We also predicted all possible homodimers and heterodimers combinations (Fig.: 4, C) to rank the sequences that form with high confidence heterodimers and have low tendency to form homodimers. We scored the designs based on pLDDT and an interface predicted align error (iPAE) (Fig.: 4, D) that is obtained by summing only over the PAE between the two chains. We experimentally tested 18 design pairs, out of which we selected three candidate design pairs because of superior expression yields. The design pair C5-C6 (Fig.: 4, E) showed two distinct monomer peaks in the SEC when expressed and purified individually. A new peak at larger molecular weight emerged in the elution profile once the purified proteins were mixed. The design pair D9-D10 also showed the designed heterodimer formation when we mixed both chains (Fig.: 4, F). Interestingly, one of the monomer chains (D9) formed a homo-oligomer that disassembled and got replaced with the hetero-dimer when D9 was mixed with D10. AF2 predicts that D9 by itself forms a homo-tetramer. These tests of homo- and heterooligomer designs, while not exhaustive, suggest that our relaxed sequencing hallucination method is also capable of designing functional protein-protein interfaces that specifically interact with each other.

Finally, we validated experimentally the structures of a set of candidate designs using single particle cryo electron microscopy (cryo-EM) analysis. To this end we randomly chose three out of the five proteins comprising 600 amino acids described in Figure 3 with a molecular weight around 60 kDa. We determined 3D cryo-EM maps for all three candidate designs using image data acquired with a Titan Krios cryo EM followed by image processing with RELION20. The resulting maps had resolution between 5.7 and 5.9 A according to the FSC gold standard criterion21 (Supp. Fig.: 4).

At this resolution, the global shape and secondary structure elements can be clearly discerned (Fig.: 5, A, B, C). Rigid body docking using ChimeraX22 of AF2 predictions to the measured cryo EM density maps showed good agreement with the designed models. While the resolution was not high enough to obtain atomic models from the densities, using PHENIX real space refinement23 we could fit the AF2 predictions to the cryoEM densities and obtain RMSDs of 0.36 A (P005), 0.44 A (P008) and 0.544 A (P009), respectively, between the PHENIX-fitted atomic models and the AF2 predictions. While these fits may not be sufficiently accurate to evaluate details such as side chain geometries, they do indicate that the relaxed sequence pipeline delivers correct output backbone geometries. 229 Conclusion 230 In this work we introduced and experimentally tested a novel backbone generation 231 method using relaxed sequence hallucination. By enabling the unconstrained 232 optimization of sequence gradients using this relaxed sequence representation, our 233 method can efficiently and accurately generate protein backbones up to 600 amino 234 acids long and possibly beyond. We demonstrated experimentally that the designs 235 translate to real proteins that fold well and have the designed properties. Our 236 approach can generate monomeric proteins but also homo oligomers and 237 heterodimers. The relaxed sequence hallucination method provides substantial 238 efficiency advantages relative to the more commonly used MCMC methods. For 239 example, recently described de-novo designed luciferase enzymes were produced 240 by MCMC hallucination with 30,000 iterations4 or large language model based 241 designs requiring up to 170.000 iterations9. By contrast, our gradient-descent 242 approach typically converged within < 100 iterations. We thus expect our method to 243 drastically speed up the throughput and scope of protein design tasks. Finally, our 244 relaxed sequence gradient descent provides an alternative to recently introduced 245 denoising diffusion models17,24–26, providing researchers with even more tools to 246 enhance the capabilities of de novo protein design. 247 248 249 250 251 252 253 254 255

C.F. and H.D. designed the research. H.D. supervised the research. C.F. designed proteins and performed computational studies. D.S and C.F. designed heterodimers. Y.d.S designed AF only proteins. S.O. and S.F. developed the ColabDesign Framework. S.O. helped conduct computational studies. C.F. Y.d.S. and D.S. performed wet lab experiments. A.K. prepared Cryo-EM grids, collected data and performed data processing. C.F., A.K. and H.D wrote the manuscript. Y.d.S. and S.O. proofread the manuscript.

All experimental data, sequences, AF2 predictions and Cryo-EM maps will be made available after publication. The exact code used in this manuscript will also be available after publication. The code for the Colabdesign framework is freely available at: https://github.com/sokrypton/ColabDesign 262 Acknowledgement 263 We thank Google Cloud Services for providing computational resources. We thank 264 Massimo Kube for help with compute cluster maintenance and useful discussions 265 and Maximilian Honemann for guidance with cloning. We thank Justas Dauparas for 266 help debugging the ColabDesign code. 267 268 This work was supported by a European Research Council Advanced Grant to H.D. 269 (grant agreement 101018465), the Deutsche Forschungsgemeinschaft through grants 270 provided within the Gottfried Wilhelm Leibniz Program (to H.D.). We (H.D. & C.F.) 271 acknowledge additional support via the Excellence Strategy of the Federal 272 Government and the Länder through the TUM Innovation Network RISE grant. S.O. 273 is supported by NIH DP5OD026389, NSF MCB2032259 and Amgen. 274

FIGURES & CAPTIONS individual protomers and assembled heterodimer pair D9 & D10. In the combined plot A is added in 1:1.2 excess. 322

Supplementary Figures

Supp. Figure 1 | Computational benchmarking A) Comparison of design trajectories between relaxed-sequence hallucination and MCMC based design. X-axis is linear scaled up until 100 steps and then logarithmic. Both relaxed sequence and MCMC was performed using the ColabDesign framework. A mutation rate of 3 was used for MCMC. B) Time measured to perform one design trajectory using the CD framework on an Nvidia A100. For logits hallucination 100 iterations were used while for MCMC 2000 iterations with a mutation rate of 3 was used. C) Three design trajectories of 200 steps of hallucination performed with the ‘hard’ protocol using ColabDesign.

Supp. Figure 2 | Additional experimental verification. A) Laser scanned images of SDS-PAGE gels analyzing unconditional monomers with 100, 200 and 300 AAs. L indicates the ladder, while the sample bands are elution fractions after IMAC. B) Laser scanned images of SDS-PAGE gels analyzing unconditional monomers with 600 AAs size. E1-E4 indicate the elution fractions after IMAC, while FT shows the flow through C) Experimental testing of AF only generated sequences. TOP: Comparison of expression yield. Yield of protein expression is determined as amount of soluble proteins measured with nanodrop after expression and IMAC purification. MIDDLE: CD spectra of MPNN redesigned backbones. BOTTOM: HPLC SEC traces of proteins indicating their monomeric status.

Supp. Figure 3 | SDS-PAGE gels of Homo Oligomers and CD-spectra A) Laser scanned images of SDS-PAGE gels showing experimental success of homo oligomer expression and IMAC purification. P = Pellet ; S = Supernatant ; FT = IMAC Flow Through ; E = IMAC elusion fraction ; L = Ladder B) CD spectra of homo oligomers C) CD-Spectra of heterodimer protomers measured only at room temperature 348 349 350

Supp. Figure 4 | FSC curves of CryoEM densities FSC Curves created using RELION 4.1 for cryoEM reconstructions shown in Fig.: 5.