Pre-trained Language Model | Yequan's Academic http://localhost:1313/tag/pre-trained-language-model/ Pre-trained Language Model Wowchemy (https://wowchemy.com)en-usWed, 03 Jul 2024 00:00:00 +0000 http://localhost:1313/media/icon_hu041b0395efa72cb92c3618e7883e8354_359622_512x512_fill_lanczos_center_3.png Pre-trained Language Model http://localhost:1313/tag/pre-trained-language-model/ 52B to 1T: Lessons Learned via Tele-FLM Series http://localhost:1313/publication/arxiv2024-tele-flm-1t/ Wed, 03 Jul 2024 00:00:00 +0000 http://localhost:1313/publication/arxiv2024-tele-flm-1t/ Masked Structural Growth for 2x Faster Language Model Pre-training http://localhost:1313/publication/iclr2024-msg/ Tue, 07 May 2024 00:00:00 +0000 http://localhost:1313/publication/iclr2024-msg/ Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales http://localhost:1313/publication/arxiv2023-mu-scaling/ Fri, 14 Apr 2023 00:00:00 +0000 http://localhost:1313/publication/arxiv2023-mu-scaling/ The overview of Big Language Model and ChatGPT http://localhost:1313/talk/the-overview-of-big-language-model-and-chatgpt/ Thu, 23 Feb 2023 09:00:00 +0000 http://localhost:1313/talk/the-overview-of-big-language-model-and-chatgpt/ <p>This talk aims to clearly reveal the challenge and the powerful abilities of ChatGPT and the used GPT series language models. More importantly, we hope to discuss the future direction of both academic and industry.</p> <p>Specifically, we introduce the language model, Wudao 2.0 and the academic research of language model. The slides used will be released after the stage of anonymous appraisal.</p>