Yifei Zou

Federating to Grow Transformers with Constrained Resources without Model Sharing

2024 · arXiv (Cornell University)

The high resource consumption of large-scale models discourages resource-constrained users from developing their customized transformers. To this end, this paper considers a federated framework named Fed-Grow for multiple participants to cooperatively scale a transformer from …