Alberto Bietti

On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes

2022 · arXiv (Cornell University)

Neural networks are known to be highly sensitive to adversarial examples. These may arise due to different factors, such as random initialization, or spurious correlations in the learning problem. To better understand these factors, we …

xVal: A Continuous Numerical Tokenization for Scientific Language Models

2023 · arXiv (Cornell University)

Due in part to their discontinuous and discrete default encodings for numbers, Large Language Models (LLMs) have not yet been commonly used to process numerically-dense scientific datasets. Rendering datasets as text, however, could help aggregate …

Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers

2024 · arXiv (Cornell University)

Large language models have been successful at tasks involving basic forms of in-context reasoning, such as generating coherent language, as well as storing vast amounts of knowledge. At the core of the Transformer architecture behind …