Collection of papers and related works for Large Language Models (ChatGPT, GPT-3, Codex etc.).
This repository is contributed by the following contributors.
- Organizers: Guilin Qi (漆桂林), Xiaofang Qi (戚晓芳)
- Paper Collectors: Zafar Ali, Sheng Bi (毕胜), Yongrui Chen (陈永锐), Zizhuo Chen (陈孜卓), Xinbang Dai (戴鑫邦), Huan Gao (高桓), Nan Hu (胡楠), Shilong Hu (胡世龙), Jingqi Kang (康婧淇), Jiaqi Li (李嘉琦), Dehai Min (闵德海), Guilin Qi (漆桂林), Yiming Tan (谭亦鸣), Tongtong Wu (吴桐桐), Songlin Zhai (翟松林), Shenyu Zhang (张沈昱), Yuxin Zhang (张裕欣)
- Maintainers: Runzhe Wang (王润哲), Shenyu Zhang (张沈昱)
The automation script of this repo is powered by Auto-Bibfile. If you'd like to commit to this repo, please modify bibtex.bib or related_works.json and re-generate README.md using python scripts/run.py.
- [Overview] -- Homepage
- -- Summary
- -- Author
- -- Techniques
- -- Published Time
- -- Published Venue
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity,
by Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji et al.本文提出了一个使用公开数据集定量评估交互式LLM（如ChatGPT）的框架。我们使用涵盖8个不同的常见NLP应用任务的21个数据集对ChatGPT进行了广泛的技术评估。我们基于这些数据集和一个新设计的多模态数据集评估了ChatGPT的多任务、多语言和多模态方面。
by Qin, Chengwei, Zhang, Aston, Zhang, Zhuosheng, Chen, Jiaao, Yasunaga, Michihiro and Yang, Diyi
by Reham Omar, Omij Mangukiya, Panos Kalnis and Essam Mansour
by Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier and Julius Berner
by Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen and Wei Cheng
by Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang et al.
by Roberto Gozalo-Brizuela and Eduardo C. Garrido-Merch'an
by Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du and Dacheng Tao
by Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen and Guilin Qi
by Ning Bian, Xianpei Han, Le Sun, Hongyu Lin, Yaojie Lu and Ben He
by Ziyu Zhuang, Qiguang Chen, Longxuan Ma, Mingda Li, Yi Han, Yushan Qian, Haopeng Bai, Zixian Feng et al.
by Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng and Tushar KhotA Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets,
by Md. Tahmid Rahman Laskar, M. Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty and Jimmy X. Huang
by Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin and Erik CambriaHolistic Evaluation of Language Models,
by Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan et al.
by Nitarshan Rajkumar, Raymond Li and Dzmitry BahdanauAre Visual-Linguistic Models Commonsense Knowledge Bases?,
by Hsiu-Yu Yang and Carina Silberer
by Xingxuan Li, Yutong Li, Linlin Liu, Lidong Bing and Shafiq R. JotyGeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models,
by Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li and Kai-Wei Chang
by Soumya Sanyal, Zeyi Liao and Xiang Ren
by Frank F. Xu, Uri Alon, Graham Neubig and Vincent J. Hellendoorn
by Arshiya Aggarwal, Jiao Sun and Nanyun PengEvaluating Large Language Models Trained on Code,
by Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond'e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda et al.GLGE: A New General Language Generation Evaluation Benchmark,
by Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu et al.
by Mohammad Abdul Hadi and Fatemeh H. FardDo Language Models Perform Generalizable Commonsense Inference?,
by Peifeng Wang, Filip Ilievski, Muhao Chen and Xiang RenRICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms,
by Pei Zhou, Rahul Khanna, Seyeon Lee, Bill Yuchen Lin, Daniel Ho, Jay Pujara and Xiang RenEvaluation of Text Generation: A Survey,
by Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao
by Cristina Garbacea and Qiaozhu MeiBERTScore: Evaluating Text Generation with BERT,
by Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger and Yoav Artzi
by Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu et al.This paper surveys and summarizes the progress and challenges of ICL, including ICL's formal definition, correlation to related studies, advanced techniques (training strategies, related analysis) and potential directions.
by Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji et al.
by Xavier Daull, Patrice Bellot, Emmanuel Bruno, Vincent Martin and Elisabeth Murisasco
by Gr'egoire Mialon, Roberto Dess`\i, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozi`ere, Timo Schick et al.
by Boxi Cao, Hongyu Lin, Xianpei Han and Le Sun
by Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang et al.Survey of Hallucination in Natural Language Generation,
by Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang et al.
by Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang et al.
by Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang et al.ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope,
by Ray, Partha Pratim
by Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo et al.
by Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang and Xindong Wu
by Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou and Ji-Rong Wen
by Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi et al.
by Lin Geng Foo, Hossein Rahmani and Jun LiuDomain specialization as the key to make large language models disruptive: A comprehensive survey,
by Ling, Chen, Zhao, Xujiang, Lu, Jiaying, Deng, Chengyuan, Zheng, Can, Wang, Junxiang, Chowdhury, Tanmoy, Li, Yun et al.
by Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu and Lichao Sun
by Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu et al.
by Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Jiayang Cheng, Yunzhi Yao, Wenyang Gao et al.
by Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu and Li Shen
by Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu et al.
by Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin et al.Large Language Models for Generative Information Extraction: A Survey,
by Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng et al.
by Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng and Jeffrey Xu Yu
by Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang and Jian-Guang LouA Survey on Knowledge-Enhanced Pre-trained Language Models,
by Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yifei Li, Yong Chen and Dell ZhangA Continual Learning Survey: Defying Forgetting in Classification Tasks,
by Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh and Tinne TuytelaarsThe survey: Text generation models in deep learning,
by Touseef Iqbal and Shaima QureshiFrom distributed machine learning to federated learning: a survey,
by Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong and Dejing DouDeep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code,
by Changan Niu, Chuanyi Li, Bin Luo and Vincent Ng
by Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong and Qing He
by Sagi Shaier, Lawrence Hunter and Katharina Kann
by Jie Huang and Kevin Chen-Chuan Chang
by Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang et al.
by Huayang Li, Yixuan Su, Deng Cai, Yan Wang and Lemao LiuCommonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey,
by Prajjwal Bhargava and Vincent Ng
by Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona T. Diab and Marjan Ghazvininejad
by Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke and Tat-Seng Chua
by Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Vinay Adiga and Erik Cambria
by Tara Safavi and Danai Koutra
by Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao
In-context Learning with Retrieved Demonstrations for Language Models: A Survey,
by Man Luo, Xin Xu, Yue Liu, Panupong Pasupat and Mehran Kazemi
by Manish Chandra, Debasis Ganguly, Yiwen Li and Iadh Ounis"In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval",
by Andrew Parry, Debasis Ganguly and Manish Chandra
by Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu et al.This paper surveys and summarizes the progress and challenges of ICL, including ICL's formal definition, correlation to related studies, advanced techniques (training strategies, related analysis) and potential directions.
by Xi Ye and Greg Durrett
by Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu and Lingpeng KongThis paper proposes a LM named EvaLM to scale up the sequence length (trained with 8k tokens per batch line). Experiments based on EvaLM prove that in-context learning can achieve higher performance with more demonstrations under many-shot instruction tuning (8k) and further extending the length of instructions (16k) can further improve the upper bound of scaling in-context learning.
by Xinyi Wang, Wanrong Zhu and William Yang Wang
by Xiaonan Li and Xipeng Qiu
by Noam Wies, Yoav Levine and Amnon Shashua
by Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim and Minjoon Seo
by Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu and Mohit Iyyer
by Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, Huazhu Fu et al.
by Man Luo, Xin Xu, Zhuyun Dai, Panupong Pasupat, Seyed Mehran Kazemi, Chitta Baral, Vaiva Imbrasaite and Vincent Y. Zhao
by Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang et al.Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Text,
by Hanieh Khorashadizadeh, Nandana Mihindukulasooriya, Sanju Tiwari, Jinghua Groppe and Sven Groppe
by Qinyong Wang, Zhenxiang Gao and Rong Xu
by Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu and Chenguang Zhu- <img src=https://img.shields.io/badge/the_61st_Annual_Meeting_of_the_Association_for_Computational
Linguistics_(Volume_2:Short_Papers),{ACL}_2023,_Toronto,_Canada,
July_9--14,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> MetaVL: Transferring In-Context Learning Ability From Language Models
to Vision-Language Models,
by Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin Yang and Kai-Wei Chang
by Yi-Syuan Chen, Yun-Zhu Song, Cheng Yu Yeo, Bei Liu, Jianlong Fu and Hong-Han Shuai
by Jiuhai Chen, Lichang Chen, Chen Zhu and Tianyi Zhou
by Chi Han, Ziqi Wang, Han Zhao and Heng JiMeta-in-context learning in large language models,
by Julian Coda-Forno, Marcel Binz, Zeynep Akata, Matt M. Botvinick, Jane X. Wang and Eric SchulzMeta-learning via Language Model In-context Tuning,
by Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis and He HeThis paper proposes in-context tuning, which recasts task adaptation and prediction as a simple sequence prediction problem: to form the input sequence, concatenate the task instruction, labeled in-context examples, and the target input to predict; to meta train the model to learn from in-context examples, finetune a PLM to predict the target label given the input sequence on a collection of tasks (very similar to MetaICL). On LAMA and BinaryClfs, the proposed method outperforms MAML.MetaICL: Learning to Learn In Context,
by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh HajishirziMetaICL proposes a supervised meta-training framework to enable LMs to more effectively learn a new task in context. In MetaICL, each meta-training example includes several training examples from one task that will be presented together as a single sequence to the LM, and the prediction of the final example is used to calculate the loss.
by Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf et al.This paper proposes a graph-based selective annotation method named vote-k to(1) select a pool of examples to annotate from unlabeled data,(2) retrieve prompts (contexts) from the annotated data pool for in-context learning.Specifically, the selection method first selects a small set of unlabeled examples iteratively and then labels them to serve as contexts for LLMs to predict the labels of the rest unlabeled data. The method selects the predictions with highest confidence (log probability of generation output) to fill up the selective annotation pool.
by Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov and Zornitsa KozarevaThis paper proposes to use self-supervision (MLM, NSP, CL, etc.) between pre-training and downstream usage to teach the LM to perform in-context learning. Analysis reveals that:(1) benefits of self-supervised depends on the amount of training data,(2) semantic similarity between training and evaluation tasks matters,(3) adding training objectives without diversity does not help,(4) model performance improves when choosing similar templates for both self-supervised and downstream tasks,(5) self-supervised tasks and human-annotated datasets are complementary,(6) self-supervised-trained models are better at following task instructions.
by Or Honovich, Uri Shaham, Samuel R. Bowman and Omer Levy(1) 探索了利用LLM在几个样本的情况下归纳出任务指令的能力；(2) 测量两个指标：1. 模型归纳指令与人类归纳的指令对比，2. 利用模型归纳的指令作为prompt进行预测的执行准确率；(3) 相比于GPT-3，InstructGPT效果更好，理所当然。
by Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel and Pontus Stenetorp(1) This work demonstrates that few-shot prompts suffer from order sensitivity, in that for the same prompt the order in which samples are provided can make a difference to model performance.(2) This work introduces a probing method which constructs an artificial development set by language models themselves to alleviate the order sensitivity problem.
by Ohad Rubin, Jonathan Herzig and Jonathan BerantThis paper proposes a method to retrieve good contexts for in-context learning. Specifically, the method(1) uses an unsupervised retriever (BM25/SBERT) to obtain a set of context candidates,(2) passes the candidates to a scoring model (GPT-Neo/GPT-J/GPT-3/Codex) and select the top/bottom k as positive/negative examples,(3) uses the examples to train a dense retriever (BERT-based).
by Yiming Zhang, Shi Feng and Chenhao Tan(1) This paper revisits the effect of example selection (re-ordering & calibration) for ICL, observing that a large variance across set of demonstration examples still exists.(2) This paper applies reinforcement learning (Q-Learning) to optimize example selection by formulating this task as sequential decision-making problem, which is appropriate for example selection from unlabeled datasets.
by Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo and Sang-goo LeeMeasuring Convergence Inertia: Online Learning in Self-adaptive Systems with Context Shifts,
by Elvin Alberts and Ilias GerostathopoulosAn Explanation of In-context Learning as Implicit Bayesian Inference,
by Sang Michael Xie, Aditi Raghunathan, Percy Liang and Tengyu Ma
by Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi and Luke Zettlemoyer
by Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing
by Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin and Weizhu Chen(1) 探索了在in-context learning中什么样的demonstration example可以对GPT-3的效果取得帮助；(2) 利用roberta对样本进行编码，并计算demonstration与test example的向量距离（欧氏距离），最终发现与test example越相近的demonstration越能取得较好的效果。Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again,
by Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun and Yu Su
by Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui and Furu Wei(1) 与The Dual Form of Neural Networks Revisited结合一起看，可以进一步理解in-context learning，通过与NN线性层对偶形式的类比，可以将ICL流程描述为：1. 基于Transformer的预训练语言模型作为元优化器；2. 通过正向计算，根据示范例子产生元梯度；3. 通过关注，将元梯度应用于原始语言模型，建立一个ICL模型；(2)与Fine-tune类似，ICL也是在zero-shot learning参数的基础上，提供了一个更新量。The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention,
by Kazuki Irie, R'obert Csord'as and J"urgen Schmidhuber(1) 很有意思的一篇，回顾神经网络（NN）线性层Y=WX（省略偏置b）的原始形式与对偶形式，两种形式完全等价；(2) 从对偶形式中可以发现，通过反向传播训练的NN线性层的输出主要是该层在训练期间的训练误差信号et的线性组合，其中权重是通过比较测试查询x和每个训练输入计算出来的；进一步可以得出，如果测试时输入的x和训练时的输入是正交的，那么梯度下降所得到的参数更新对于该样本x完全没有影响。
by Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye and Lingpeng Kong
by Ting-Yun Chang and Robin Jia
by Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff and Dan Roth
by Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai et al.Can Large Language Models Understand Real-World Complex Instructions?,
by Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang et al.
by Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Sch"arli and Denny Zhou
by Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini et al.
by Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee and Minjoon Seo
by Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li et al.
by Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley and Jianfeng Gao
by Haotian Liu, Chunyuan Li, Qingyang Wu and Yong Jae Lee
by Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung et al.
by Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li and Ying Shan
by Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Xiaoshui Huang, Zhiyong Wang et al.
by Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai et al.
by Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos et al.
by Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang et al.
by Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran et al.
by Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi
by Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar and Chitta Baral
by Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini et al.
by Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh et al.
by Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin et al.
by Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, J'er'emy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak et al.
by Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune and Abhinav Rastogi
by Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort et al.
by Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kiant'e Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi and Yejin Choi
by Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, H. Francis Song, Martin Chadwick, Mia Glaese, Susannah Young et al.
by Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger et al.
by Gao, Leo, Schulman, John and Hilton, Jacob
by Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez et al.
by Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Ido Greenberg, Avinatan Hassidim, Michael Fink, Yossi Matias et al.
by Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal et al.Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences,
by Erdem Biyik, Dylan P. Losey, Malayandi Palan, Nicholas C. Landolfi, Gleb Shevchuk and Dorsa Sadigh
by Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain et al.
by Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike and Paul F. ChristianoLearning to summarize with human feedback,
by Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei et al.Dialogue Response Ranking Training with Large-Scale Human Feedback Data,
by Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett and Bill DolanFine-Tuning Language Models from Human Preferences,
by Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano and Geoffrey IrvingDeep Reinforcement Learning from Human Preferences,
by Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg and Dario Amodei
GPT-4 Technical Report,
by OpenAI
by OpenAI
by Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab et al.
by Hui Su, Xiao Zhou, Houjin Yu, Yuwen Chen, Zilin Zhu, Yang Yu and Jie Zhou
by Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam et al.
by Kevin Clark, Minh-Thang Luong, Quoc V. Le and Christopher D. ManningRevisiting Pre-Trained Models for Chinese Natural Language Processing,
by Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang and Guoping Hu
by Pengcheng He, Xiaodong Liu, Jianfeng Gao and Weizhu ChenExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,
by Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li et al.A Primer in BERTology: What We Know About How BERT Works,
by Anna Rogers, Olga Kovaleva and Anna RumshiskyLanguage Models are Unsupervised Multitask Learners,
by Radford, Alec, Wu, Jeffrey, Child, Rewon, Luan, David, Amodei, Dario and Sutskever, IlyaBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,
by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova
by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis et al.Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,
by Nils Reimers and Iryna GurevychImproving language understanding by generative pre-training,
by Radford, Alec, Narasimhan, Karthik, Salimans, Tim, Sutskever, Ilya and others
by Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du et al.
by Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen
by Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani et al.
by Tao Zhong, Zhixiang Chi, Li Gu, Yang Wang, Yuanhao Yu and Jin Tang
by Xintao Wang, Zhouhong Gu, Jiaqing Liang, Dakuan Lu, Yanghua Xiao and Wei Wang
by Tanmoy Chowdhury, Chen Ling, Xuchao Zhang, Xujiang Zhao, Guangji Bai, Jian Pei, Haifeng Chen and Liang ZhaoDeep Bidirectional Language-Knowledge Graph Pretraining,
by Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang and Jure Leskovec
by Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yifei Li, Yong Chen and Dell ZhangReview of Knowledge-Enhanced Pre-trained Language Models,
by Yi, HAN, Linbo, QIAO, Dongsheng, LI and Xiangke, LIAO
by Sagi Shaier, Lawrence Hunter and Katharina Kann
by Kesong Liu, Jianhui Jiang and Feifei Lyu
by Jianing Wang, Wenkang Huang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Xiang Li and Ming GaoDict-BERT: Enhancing Language Model Pre-training with Dictionary,
by Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng and Meng Jiang
by Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning and Jure LeskovecSPOT: Knowledge-Enhanced Language Representations for Information Extraction,
by Jiacheng Li, Yannis Katsis, Tyler Baldwin, Ho-Cheol Kim, Andrew Bartko, Julian J. McAuley and Chun-Nan Hsu
by Jinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeon-Jin Park, Ji-Hoon Kim, Jisu Jeong, Kyung-Min Kim and Hyunwoo J. KimKnowledge-based Review Generation by Coherence Enhanced Text Planning,
by Junyi Li, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
by Shilei Liu, Xiaofeng Zhao, Bochao Li, Feiliang Ren, Longhui Zhang and Shujuan YinAsk what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge,
by Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley and Julian J. McAuley
by Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen et al.
by Zheng Yuan, Yijia Liu, Chuanqi Tan, Songfang Huang and Fei Huang
by Wenhu Chen, Yu Su, Xifeng Yan and William Yang Wang
by Jian Guan, Fei Huang, Minlie Huang, Zhihao Zhao and Xiaoyan ZhuKnowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network,
by Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
by Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar and Bryan CatanzaroBarack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling,
by Robert L. Logan IV, Nelson F. Liu, Matthew E. Peters, Matt Gardner and Sameer SinghZero-shot Learning with Semantic Output Codes,
by Mark Palatucci, Dean Pomerleau, Geoffrey E. Hinton and Tom M. Mitchell
by Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee et al.Compressing Pre-trained Models of Code into 3 MB,
by Jieke Shi, Zhou Yang, Bowen Xu, Hong Jin Kang and David Lo
by Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen and Tuo Zhao
by Tao Zhong, Zhixiang Chi, Li Gu, Yang Wang, Yuanhao Yu and Jin Tang
by Chenhao Wang, Jiachun Li, Yubo Chen, Kang Liu and Jun Zhao
by Dongha Choi, Hongseok Choi and Hyunju LeePreparing lessons: Improve knowledge distillation with better supervision,
by Tiancheng Wen, Shenqi Lai and Xueming Qian
by Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong and Furu Wei
by Shizhe Diao, Ruijia Xu, Hongjin Su, Yilei Jiang, Yan Song and Tong ZhangDistilling Knowledge Learned in BERT for Text Generation,
by Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu and Jingjing LiuImproved Knowledge Distillation via Teacher Assistant,
by Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa and Hassan GhasemzadehRegularizing Class-Wise Predictions via Self-Knowledge Distillation,
by Sukmin Yun, Jongjin Park, Kimin Lee and Jinwoo ShinRelational Knowledge Distillation,
by Wonpyo Park, Dongju Kim, Yan Lu and Minsu Cho
by Li Yuan, Francis E. H. Tay, Guilin Li, Tao Wang and Jiashi FengKnowledge Distillation via Route Constrained Optimization,
by Xiao Jin, Baoyun Peng, Yichao Wu, Yu Liu, Jiaheng Liu, Ding Liang, Junjie Yan and Xiaolin Hu
by Elahe Arani, Fahad Sarfraz and Bahram Zonooz
by Raphael Tang, Yao Lu, Linqing Liu, Lili Mou, Olga Vechtomova and Jimmy LinThe Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks,
by Jonathan Frankle and Michael CarbinBorn-Again Neural Networks,
by Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti and Anima AnandkumarPaying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer,
by Sergey Zagoruyko and Nikos Komodakis
by Antti Tarvainen and Harri ValpolaDeep Mutual Learning,
by Ying Zhang, Tao Xiang, Timothy M. Hospedales and Huchuan LuDeep Model Compression: Distilling Knowledge from Noisy Teachers,
by Bharat Bhusan Sau and Vineeth N. BalasubramanianDistilling the Knowledge in a Neural Network,
by Geoffrey E. Hinton, Oriol Vinyals and Jeffrey Dean
Crawling the Internal Knowledge-Base of Language Models,
by Roi Cohen, Mor Geva, Jonathan Berant and Amir Globerson本文提出一种从语言模型中提取结构化知识图谱的方法；使用专门设计的提示来控制提取过程中的精度和召回率；在GPT-3上进行了评估，显示了高精确度的结果。
by Mehran Kazemi, Sid Mittal and Deepak Ramachandran
by Hamed Babaei Giglou, Jennifer D'Souza and S"oren Auer
by Simon Gottschalk and Elena Demidova
by Hongbin Ye, Ningyu Zhang, Hui Chen and Huajun Chen
by Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui and Lei Li
by Benjamin Newman, Prafulla Kumar Choubey and Nazneen Rajani
by Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi and Hannaneh Hajishirzi
by Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi and Yejin Choi
by Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck et al.
by Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West et al.
by Xinbei Ma, Tianjie Ju, Jiyang Qiu, Zhuosheng Zhang, Hai Zhao, Lifeng Liu and Yulong Wang
by Xiaobao Wu, Liangming Pan, William Yang Wang and Anh Tuan Luu
by Hao Peng, Xiaozhi Wang, Chunyang Li, Kaisheng Zeng, Jiangshan Duo, Yixin Cao, Lei Hou and Juanzi Li
by Davis Brown, Charles Godfrey, Cody Nizinski, Jonathan Tu and Henry Kvinge
by Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong and Zhang Xiong
by Siyuan Cheng, Ningyu Zhang, Bozhong Tian, Zelin Dai, Feiyu Xiong, Wei Guo and Huajun Chen
by Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma and Jie Yu
by Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts and Danqi Chen
by Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas and Fazl Barez
by Peter Hase, Mona T. Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal and Srinivasan Iyer
by Yucheng Shi, Shaochen Xu, Zhengliang Liu, Tianming Liu, Xiang Li and Ninghao Liu
by Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn and Christopher D. Manning
by Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning and Chelsea Finn
by Meng, Kevin, Bau, David, Andonian, Alex J and Belinkov, Yonatan
by Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim and Marzyeh Ghassemi
by Nicola De Cao, Wilker Aziz and Ivan Titov
by Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva and Sebastian RiedelTowards Systematic Evaluation of Logical Reasoning Ability of Large Language Models,
by Anonymous Submission
by Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han and Yongbin Li
by Boshi Wang, Xiang Yue, Yu Su and Huan Sun
by Simon Ott, Konstantin Hebenstreit, Valentin Li'evin, Christoffer Egeberg Hother, Milad Moradi, Maximilian Mayrhauser, Robert Praas, Ole Winther et al.
by Tanmoy Chowdhury, Chen Ling, Xuchao Zhang, Xujiang Zhao, Guangji Bai, Jian Pei, Haifeng Chen and Liang Zhao
by Yunhu Ye, Binyuan Hui, Min Yang, Binhua Li, Fei Huang and Yongbin Li
by Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal and Tushar Khot
by Imani, Shima, Du, Liang and Shrivastava, Harsh
by Fangzhen Lin, Ziyi Shou and Chengcai chenComplexity-Based Prompting for Multi-step Reasoning,
by Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark and Tushar Khot
by Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen R. McKeown, Doug Downey and Yejin Choi
by Phillip Howard, Junlin Wang, Vasudev Lal, Gadi Singer, Yejin Choi and Swabha Swayamdipta
by Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li and Yanghua Xiao
by Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu and Erik Cambria
by Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang and Dongsheng Li
by Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju, Murray Shanahan, Miguel L'azaro-Gredilla and Dileep George
by Zhang, Hanlin, Li, Ziyang, Huang, Jiani, Naik, Mayur and Xing, Eric
by Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord et al.
by Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras and Yejin Choi
by Swarnadeep Saha, Xinyan Velocity Yu, Mohit Bansal, Ramakanth Pasunuru and Asli Celikyilmaz
by Wenhu Chen, Xueguang Ma, Xinyi Wang and William W. Cohen
by Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing
by Jie Huang and Kevin Chen-Chuan Chang
by Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen and Xiaodan Liang
by Denny Zhou, Nathanael Sch"arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet et al.(1) 两阶段的prompt，第一阶段问题分解（通过in-context learning实现，context中包含了其他问题的分解示例），对于每个问题，分解出回答该问题需要先回答什么子问题；(2) 在第二阶段中，从后往前依次解决子问题，同样通过in-context learing得到，每次LLM的回答会参与组成下一个问题的prompt。
by Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi and Denny Zhou
by Ye, Xi and Durrett, GregJiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding,
by Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang et al.
by Batu Ozturkler, Nikolay Malkin, Zhen Wang and Nebojsa Jojic
by Soumya Sanyal, Zeyi Liao and Xiang Ren
by Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil et al.
by Yeliang Xiu, Zhanhao Xiao and Yongmei Liu
by Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith and Yejin Choi
by Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwaśniewski et al.
by Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma and Yitao Liang
by Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui and Furu Wei
by Sijia Chen, Baochun Li and Di Niu
by Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund and Chuang Gan
by Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis and Alex Smola
by Hangfeng He, Hongming Zhang and Dan Roth本文通过用GPT-3在三个复杂的推理任务：常识推理，时间推理和表格推理上进行大量实验来评估RR的有效性。结果表明，RR可以产生更忠实的解释，并提高LLM的性能。
by Shizhe Diao, Pengcheng Wang, Yong Lin and Tong Zhang
by Kashun Shum, Shizhe Diao and Tong Zhang
by Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski et al.
by Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee and Ee-Peng Lim
by Jieyi Long
by Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay et al.
by Yao Yao, Zuchao Li and Hai Zhao
by Fanglong Yao, Changyuan Tian, Jintao Liu, Zequn Zhang, Qing Liu, Li Jin, Shuchao Li, Xiaoyu Li et al.
by Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji and Ajay Divakaran
by Bin Lei, Pei-Hung Lin, Chunhua Liao and Caiwen Ding
by Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao and Karthik Narasimhan
by Pengbo Hu, Ji Qi, Xingyu Li, Hong Li, Xinqi Wang, Bing Quan, Ruiyu Wang and Yi Zhou
by Peiyi Wang, Lei Li, Liang Chen, Feifan Song, Binghuai Lin, Yunbo Cao, Tianyu Liu and Zhifang Sui
by Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou and Weizhu Chen
by Anni Zou, Zhuosheng Zhang, Hai Zhao and Xiangru Tang
by Dingjun Wu, Jing Zhang and Xinmei Huang
by Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou and Sibei Yang
by Jiaxin Ge, Hongyin Luo, Siyuan Qian, Yulu Gan, Jie Fu and Shanghang Zhang
by Yingcong Li, Kartik Sreenivasan, Angeliki Giannou, Dimitris Papailiopoulos and Samet Oymak
by Or Honovich, Uri Shaham, Samuel R. Bowman and Omer Levy(1) 探索了利用LLM在几个样本的情况下归纳出任务指令的能力；(2) 测量两个指标：1. 模型归纳指令与人类归纳的指令对比，2. 利用模型归纳的指令作为prompt进行预测的执行准确率；(3) 相比于GPT-3，InstructGPT效果更好，理所当然。
by Boshi Wang, Xiang Deng and Huan Sun(1) 提出了一种迭代式的prompt-tuning方法，他们认为soft prompt应该带有语境，即在自回归解码时不同时刻应该有不同的prompt向量；(2) 利用BERT为encoder-decoder架构的PLM生成prompt，在每个解码时刻BERT都会根据先前时刻的上下文生成一组新的prompt向量，提供给PLM生成新的上下文，迭代往复。
by Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark and Tushar Khot
by Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith and Mike Lewis
by Zhuosheng Zhang, Aston Zhang, Mu Li and Alex Smola
by Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le and Denny Zhou
by Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi and Denny Zhou
by Aman Madaan and Amir Yazdanbakhsh
by Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer and Huan Sun
by Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung et al.
by Seyed Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu and Deepak Ramachandran
by Zelikman, Eric, Mu, Jesse, Goodman, Noah D and Wu, Yuhuai Tony
by Abulhair Saparov and He He
by Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo and Yusuke Iwasawa
by Antonia Creswell, Murray Shanahan and Irina Higgins
by Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma et al.
by Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang et al.
by Namgyu Ho, Laura Schmid and Se-Young Yun
by Yixuan Weng, Minjun Zhu, Shizhu He, Kang Liu and Jun Zhao
by Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang et al.
by Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan and Graham Neubig
by Wenhu Chen
by Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu and Jiawei Han
by Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal and Tushar Khot
by Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark and Tushar Khot
by Swarnadeep Saha, Xinyan Velocity Yu, Mohit Bansal, Ramakanth Pasunuru and Asli Celikyilmaz
by Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi and Denny Zhou
by Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord et al.
by Wenhu Chen, Xueguang Ma, Xinyi Wang and William W. Cohen
by Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang et al.
by Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil et al.
by Anonymous Submission
by Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han and Yongbin Li
by Boshi Wang, Xiang Yue, Yu Su and Huan Sun
by Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen R. McKeown, Doug Downey and Yejin Choi
by Phillip Howard, Junlin Wang, Vasudev Lal, Gadi Singer, Yejin Choi and Swabha Swayamdipta
by Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li and Yanghua Xiao
by Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu and Erik Cambria
by Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang and Dongsheng Li
by Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju, Murray Shanahan, Miguel L'azaro-Gredilla and Dileep George
by Zhang, Hanlin, Li, Ziyang, Huang, Jiani, Naik, Mayur and Xing, Eric
by Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras and Yejin Choi
by Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing
by Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen and Xiaodan Liang
by Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith and Yejin Choi
by Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz and Jason Weston
Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles,
by Ye, Shuquan, Xie, Yujia, Chen, Dongdong, Xu, Yichong, Yuan, Lu, Zhu, Chenguang and Liao, Jing
by Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang and Zhaocheng Zhu
by Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang and Jure Leskovec
Fairness and accuracy in horizontal federated learning,
by Wei Huang, Tianrui Li, Dexian Wang, Shengdong Du, Junbo Zhang and Tianqiang HuangFederated Learning Meets Multi-Objective Optimization,
by Zeou Hu, Kiarash Shaloudegi, Guojun Zhang and Yaoliang Yu
by Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong and Dejing Dou
by Mingyang Chen, Wen Zhang, Zhen Yao, Xiangnan Chen, Mengxiao Ding, Fei Huang and Huajun Chen
by Yun-Wei Chu, Seyyedali Hosseinalipour, Elizabeth Tenorio, Laura M. Cruz Castro, Kerrie A. Douglas, Andrew Lan and Christopher G. Brinton
by Orion Weller, Marc Marone, Vladimir Braverman, Dawn J. Lawrie and Benjamin Van DurmeRethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning,
by Liangqiong Qu, Yuyin Zhou, Paul Pu Liang, Yingda Xia, Feifei Wang, Ehsan Adeli, Li Fei-Fei and Daniel L. RubinFedBERT: When Federated Learning Meets Pre-training,
by Yuanyishu Tian, Yao Wan, Lingjuan Lyu, Dezhong Yao, Hai Jin and Lichao Sun
by John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi and Michael RabbatDitto: Fair and Robust Federated Learning Through Personalization,
by Tian Li, Shengyuan Hu, Ahmad Beirami and Virginia Smith
by Gary Cheng, Karan N. Chadha and John C. DuchiFederated Learning with Fair Averaging,
by Zheng Wang, Xiaoliang Fan, Jianzhong Qi, Chenglu Wen, Cheng Wang and Rongshan YuCollaborative Fairness in Federated Learning,
by Lingjuan Lyu, Xinyi Xu, Qian Wang and Han YuFederated Visual Classification with Real-World Data Distribution,
by Tzu-Ming Harry Hsu, Hang Qi and Matthew Brown
Distributed Training of Knowledge Graph Embedding Models using Ray,
by Nasrullah Sheikh, Xiao Qin, Yaniv Gur and Berthold ReinwaldDistributed Learning With Sparsified Gradient Differences,
by Yicheng Chen, Rick S. Blum, Martin Tak'ac and Brian M. SadlerGraph Attention Neural Network Distributed Model Training,
by Esmaeilzadeh, Armin, Zadeh Nojoo Kambar, Mina Esmail and Heidari, MaryamElastic Deep Learning Using Knowledge Distillation with Heterogeneous Computing Resources,
by Daxiang Dong, Ji Liu, Xi Wang, Weibao Gong, An Qin, Xingjian Li, Dianhai Yu, Patrick Valduriez et al.GRACE: A Compressed Communication Framework for Distributed Machine Learning,
by Hang Xu, Chen-Yu Ho, Ahmed M. Abdelmoniem, Aritra Dutta, El Houcine Bergou, Konstantinos Karatsenidis, Marco Canini and Panos KalnisLoad Balancing Optimization for Transformer in Distributed Environment,
by Delu Ma, Zhou Lei, Shengbo Chen and Peng WangDistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs,
by Da Zheng, Chao Ma, Minjie Wang, Jinjing Zhou, Qidong Su, Xiang Song, Quan Gan, Zheng Zhang et al.PyTorch Distributed: Experiences on Accelerating Data Parallel Training,
by Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith et al.Ray: A Distributed Framework for Emerging AI Applications,
by Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang et al.
by Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf et al.This paper proposes a graph-based selective annotation method named vote-k to(1) select a pool of examples to annotate from unlabeled data,(2) retrieve prompts (contexts) from the annotated data pool for in-context learning.Specifically, the selection method first selects a small set of unlabeled examples iteratively and then labels them to serve as contexts for LLMs to predict the labels of the rest unlabeled data. The method selects the predictions with highest confidence (log probability of generation output) to fill up the selective annotation pool.Selective Data Acquisition in the Wild for Model Charging,
by Chengliang Chai, Jiabin Liu, Nan Tang, Guoliang Li and Yuyu Luo
by Terry Yue Zhuo, Zhuang Li, Yujin Huang, Yuan-Fang Li, Weiqing Wang, Gholamreza Haffari and Fatemeh ShiriCodeT5Mix: A Pretrained Mixture of Encoder-decoder Transformers for Code Understanding and Generation,
by Wang, Yue, Le, Hung, Gotmare, Akhilesh Deepak, Li, Junnan and Hoi, Steven
by Zhou, Shuyan, Alon, Uri, Agarwal, Sumit and Neubig, Graham
by Xingyao Wang, Sha Li and Heng Ji
by Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang and Graham Neubig
by Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang and Jian-Guang Lou
by Nitarshan Rajkumar, Raymond Li and Dzmitry BahdanauAn extensive study on pre-trained models for program understanding and generation,
by Zhengran Zeng, Hanzhuo Tan, Haotian Zhang, Jing Li, Yuqun Zhang and Lingming Zhang
by Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese and Steven C. H. Hoi
by Jiyang Zhang, Sheena Panthaplackel, Pengyu Nie, Junyi Jessy Li and Milos Gligoric
by Jieke Shi, Zhou Yang, Bowen Xu, Hong Jin Kang and David LoDiet code is healthy: simplifying programs for pre-trained models of code,
by Zhaowei Zhang, Hongyu Zhang, Beijun Shen and Xiaodong Gu
by Saikat Chakraborty, Toufique Ahmed, Yangruibo Ding, Premkumar T. Devanbu and Baishakhi RayJigsaw: Large Language Models meet Program Synthesis,
by Naman Jain, Skanda Vaidyanath, Arun Shankar Iyer, Nagarajan Natarajan, Suresh Parthasarathy, Sriram K. Rajamani and Rahul Sharma
by Zhou Yang, Jieke Shi, Junda He and David LoAutomatic Generation of Programming Exercises and Code Explanations Using Large Language Models,
by Sami Sarsa, Paul Denny, Arto Hellas and Juho Leinonen
by Frank F. Xu, Uri Alon, Graham Neubig and Vincent J. Hellendoorn
by Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond'e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda et al.
by Yue Wang, Weishi Wang, Shafiq R. Joty and Steven C. H. HoiCodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation,
by Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain et al.
by Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray and Kai-Wei ChangTraceability Transformed: Generating more Accurate Links with Pre-Trained BERT Models,
by Jinfeng Lin, Yalin Liu, Qingkai Zeng, Meng Jiang and Jane Cleland-HuangIntelliCode compose: Code Generation using transformer,
by Alexey Svyatkovskiy, Shao Kun Deng, Shengyu Fu and Neel SundaresanMulti-task Learning based Pre-trained Language Model for Code Completion,
by Fang Liu, Ge Li, Yunfei Zhao and Zhi Jin
by Xin Wang, Yasheng Wang, Yao Wan, Jiawei Wang, Pingyi Zhou, Li Li, Hao Wu and Jin Liu
by Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou and Jian Yin
by Jos'e Antonio Hern'andez L'opez, Martin Weyssow, Jes'us S'anchez Cuadrado and Houari A. SahraouiGraphCodeBERT: Pre-training Code Representations with Data Flow,
by Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan et al.
by Xin Wang, Yasheng Wang, Pingyi Zhou, Fei Mi, Meng Xiao, Yadao Wang, Li Li, Xiao Liu et al.
by Wei Yuan, Quanjun Zhang, Tieke He, Chunrong Fang, Nguyen Quoc Viet Hung, Xiaodong Hao and Hongzhi Yin
by Nghi Bui, Yue Wang and Steven C. H. HoiMulti-view Pre-trained Model for Code Vulnerability Identification,
by Xuxiang Jiang, Yinhao Xiao, Jun Wang and Wei Zhang
by M'ark Lajk'o, Viktor Csuvik and L'aszl'o Vid'acs
by Agnieszka Ciborowska and Kostadin DamevskiApplying CodeBERT for Automated Program Repair of Java Simple Bugs,
by Ehsan Mashhadi and Hadi HemmatiA model with iterative trials for correcting logic errors in source code,
by Matsumoto, Taku, Watanobe, Yutaka and Nakamura, Keita
by Dawn Drain, Colin B. Clement, Guillermo Serrato and Neel Sundaresan
by Lingwei Li, Li Yang, Huaxi Jiang, Jun Yan, Tiejian Luo, Zihan Hua, Geng Liang and Chun Zuo
by Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green et al.
by Deze Wang, Zhouyang Jia, Shanshan Li, Yue Yu, Yun Xiong, Wei Dong and Xiangke Liao
by Rosalia Tufano, Simone Masiero, Antonio Mastropaolo, Luca Pascarella, Denys Poshyvanyk and Gabriele Bavota
by Yao Wan, Wei Zhao, Hongyu Zhang, Yulei Sui, Guandong Xu and Hai Jin
by Yiduo Guo, Yaobo Liang, Chenfei Wu, Wenshan Wu, Dongyan Zhao and Nan Duan
by Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang et al.
by Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu and Jianfeng Gao
by Changan Niu, Chuanyi Li, Bin Luo and Vincent Ng
by Yao Li, Tao Zhang, Xiapu Luo, Haipeng Cai, Sen Fang and Dawei Yuan
by Mohammad Abdul Hadi and Fatemeh H. FardWhat do pre-trained code models know about code?,
by Anjan Karmakar and Romain RobbesSentiment analysis for software engineering: How far can pre-trained transformer models go?,
by Zhang, Ting, Xu, Bowen, Thung, Ferdian, Haryono, Stefanus Agus, Lo, David and Jiang, Lingxiao
by OpenAI
by OpenAI
by Derek Chen, Celine Lee, Yunan Lu, Domenic Rosati and Zhou Yu
by Tanya Goyal, Junyi Jessy Li and Greg Durrett
by Fredrik Carlsson, Joey "Ohman, Fangyu Liu, Severine Verlinden, Joakim Nivre and Magnus Sahlgren
by Touseef Iqbal and Shaima Qureshi
by Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Mohammad Shoeybi and Bryan Catanzaro
by Rakesh Chada and Pradeep Natarajan
by Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang and Jie Tang
by Leonardo F. R. Ribeiro, Jonas Pfeiffer, Yue Zhang and Iryna Gurevych
by Jing Gu, Qingyang Wu, Chongruo Wu, Weiyan Shi and Zhou YuDialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances,
by Xiaodong Gu, Kang Min Yoo and Jung-Woo Ha
by Xinyu Hua, Ashwin Sreevatsa and Lu Wang
by Xinting Huang, Jianzhong Qi, Yu Sun and Rui Zhang
by Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu and Minlie Huang
by Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu et al.
by Junyi Li, Tianyi Tang, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
by Junyi Li, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
by Xiang Lisa Li and Percy Liang
by Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu et al.
by Shilei Liu, Xiaofeng Zhao, Bochao Li, Feiliang Ren, Longhui Zhang and Shujuan Yin
by Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang and Luo Si
by Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley and Julian J. McAuley
by Kaushal Kumar Maurya, Maunendra Sankar Desarkar, Yoshinobu Kano and Kumari Deepshikha
by Leonardo F. R. Ribeiro, Yue Zhang and Iryna Gurevych
by Lya Hulliyyatus Suadaa, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura and Hiroya Takamura
by Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen et al.
by Bowen Tan, Zichao Yang, Maruan Al-Shedivat, Eric P. Xing and Zhiting HuConsistency and Coherency Enhanced Story Generation,
by Wei Wang, Piji Li and Hai-Tao Zheng
by Xinyu Xing and Xiaojun Wan
by Xinnuo Xu, Guoyin Wang, Young-Bum Kim and Sungjin Lee
by Lanqing Xue, Kaitao Song, Duocai Wu, Xu Tan, Nevin L. Zhang, Tao Qin, Wei-Qiang Zhang and Tie-Yan Liu
by Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui et al.
by Yan Zeng and Jian-Yun Nie
by Xueying Zhang, Yunjiang Jiang, Yue Shang, Zhaomeng Cheng, Chi Zhang, Xiaochuan Fan, Yun Xiao and Bo Long
by Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam et al.
by Siqi Bao, Huang He, Fan Wang, Hua Wu and Haifeng Wang
by Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao
by Wenhu Chen, Yu Su, Xifeng Yan and William Yang Wang
by Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu and Jingjing Liu
by Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan and William Yang Wang
by Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao and Heyan Huang
by Cristina Garbacea and Qiaozhu MeiTableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching,
by Heng Gong, Yawei Sun, Xiaocheng Feng, Bing Qin, Wei Bi, Xiaojiang Liu and Ting Liu
by Jian Guan, Fei Huang, Minlie Huang, Zhihao Zhao and Xiaoyan Zhu
by Hamza Harkous, Isabel Groves and Amir Saffari
by Kalpesh Krishna, John Wieting and Mohit Iyyer
by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov and Luke Zettlemoyer
by Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
by Piji Li, Haisong Zhang, Xiaojiang Liu and Shuming Shi
by Huaishao Luo, Lei Ji, Botian Shi, Haoyang Huang, Nan Duan, Tianrui Li, Xilin Chen and Ming Zhou
by Manuel Mager, Ram'on Fernandez Astudillo, Tahira Naseem, Md. Arafat Sultan, Young-Suk Lee, Radu Florian and Salim Roukos
by Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng and Jianfeng Gao
by Hannah Rashkin, Asli Celikyilmaz, Yejin Choi and Jianfeng Gao
by Leonardo F. R. Ribeiro, Martin Schmitt, Hinrich Sch"utze and Iryna Gurevych
by Sascha Rothe, Shashi Narayan and Aliaksei Severyn
by Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang and Bo Li
by Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar and Bryan Catanzaro
by Ze Yang, Wei Wu, Can Xu, Xinnian Liang, Jiaqi Bai, Liran Wang, Wei Wang and Zhoujun Li
by Yan Zeng and Jian-Yun Nie
by Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger and Yoav Artzi
by Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu et al.
by Radford, Alec, Wu, Jeffrey, Child, Rewon, Luan, David, Amodei, Dario and Sutskever, IlyaUnified Language Model Pre-training for Natural Language Understanding and Generation,
by Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou et al.
by Sergey Golovanov, Rauf Kurbanov, Sergey I. Nikolenko, Kyryl Truskovskyi, Alexander Tselousov and Thomas Wolf
by Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian J. McAuley and Garrison W. CottrellMASS: Masked Sequence to Sequence Pre-training for Language Generation,
by Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu and Tie-Yan LiuGenerating Wikipedia by Summarizing Long Sequences,
by Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser and Noam Shazeer
by Radford, Alec, Narasimhan, Karthik, Salimans, Tim, Sutskever, Ilya and others
by Ximing Lu, Sean Welleck, Liwei Jiang, Jack Hessel, Lianhui Qin, Peter West, Prithviraj Ammanabrolu and Yejin Choi
by Shuyang Cao and Lu Wang
by Muhammad Khalifa, Hady Elsahar and Marc Dymetman
by Damian Pascual, Beni Egressy, Clara Meister, Ryan Cotterell and Roger Wattenhofer
by Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam, Seyeon Lee and Xiang Ren
by Yufei Wang, Ian D. Wood, Stephen Wan, Mark Dras and Mark Johnson
by Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski and Rosanne Liu
by Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi and Xiang Ren
by Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong and Richard Socher
by Mingxu Tao, Yansong Feng and Dongyan Zhao
by Kshitij Gupta, Benjamin Th'erien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish and Timoth'ee Lesort
by Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim and Bing Liu
by Ze-Feng Gao, Kun Zhou, Peiyu Liu, Wayne Xin Zhao and Ji-Rong WenSpeciality vs Generality: An Empirical Study on Catastrophic Forgetting in Fine-tuning Foundation Models,
by Lin, Yong, Tan, Lu, Lin, Hangyu, Zheng, Zeming, Pi, Renjie, Zhang, Jipeng, Diao, Shizhe, Wang, Haoxiang et al.
by Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot et al.
by Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh and Tinne Tuytelaars
by Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun and Jie Zhou
by Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang-Wen Li, Xiaokai Wei, Andrew O. Arnold and Xiang RenTowards Continual Reinforcement Learning: A Review and Perspectives,
by Khimya Khetarpal, Matthew Riemer, Irina Rish and Doina Precup
by Zheng Gong, Kun Zhou, Xin Zhao, Jing Sha, Shijin Wang and Ji-Rong Wen
by Chengwei Qin and Shafiq JotyWe define a challenging yet practical problem as Lifelong Few-shot Language Learning and propose a unified framework for it based on prompt tuning of T5.
by Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun KIM, Stanley Jungkyu Choi and Minjoon SeoWe propose a novel continual learning formulation named Continual Knowledge Learning which allows large language models to constantly obtain new and updated knowledge while mitigating forgetting of previous learned time-invariant knowledge.
by Tongtong Wu, Massimo Caccia, Zhuang Li, Yuan-Fang Li, Guilin Qi and Gholamreza HaffariTo explore the layer-wise property of pretrained languge models in continual learning, we thoroughly compare the continual learning performance over the combination of 5 PLMs and 4 veins of CL methods on 3 benchmarks in 2 typical incremental settings.
by Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim and Minjoon Seo
by Adam Liska, Tom'as Kocisk'y, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, Cyprien de Masson d'Autume, Tim Scholtes et al.
by Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu and Lei ShuNeurIPS 2021, The key component of CTR is the CL-plugin inserted in BERT. A CL-plugin is a capsule network with a new transfer routing mechanism to encourage knowledge transfer among tasks and also to isolate task-specific knowledge to avoid forgetting.Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning,
by Jin, Xisen , Lin, Bill Yuchen , Rostami, Mohammad and Ren, XiangWe present a new learning setup, Continual Learning of Few-Shot Learners, to address challenges of both learning settings in a unified setup, with a hyper-network for task-specific adapter generation.Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models,
by Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James R. Glass and Fuchun PengOur major finding is that after standard finetuning, the model forgets some of the important language generation skills acquired during large-scale pretraining. We propose an intuitive finetuning strategy named “mix-review”: : For each finetuning epoch, we mix the target dialogue data with a random subset of the pretraining data, mix_ratio is 4, decay is 0.9.
by Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang et al.We propose KADAPTER, a framework that retains the original parameters of the pre-trained model fixedand supports the development of versatileknowledge-infused model.
by Liu, Qingbin , Cao, Pengfei , Liu, Cao , Chen, Jiansong , Cai, Xunliang , Yang, Fan , He, Shizhu , Liu, Kang et al.This paper explores Domain-Lifelong Learning for Dialogue State Tracking, we propose Knowledge Preservation Network, which consists of multi-prototype enhanced retrospection and multi-strategy knowledge distillation, to solve the problems of expression diversity and combinatorial explosion in the DLL-DST task
by Ke, Zixuan , Liu, Bing , Xu, Hu and Shu, LeiThe key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing.
by Situ, Xuelin , Maruf, Sameen , Zukerman, Ingrid , Paris, Cecile and Haffari, GholamrezaWe propose a novel Lifelong Explanation approach that continuously trains a student explainer under the supervision of a teacher – an arbitrary explanation algorithm – on different tasks undertaken in LL. We also leverage the Experience Replay mechanism to prevent catastrophic forgetting in the student explainer.
by Yingzhu Zhao, Chongjia Ni, Cheung-Chi Leung, Shafiq R. Joty, Eng Siong Chng and Bin MaPrefix-based user identifier, Continual ASR / Architecture Search / Network Pruning.Dynamic Language Models for Continuously Evolving Content,
by Amba Hombaiah, Spurthi, Chen, Tao, Zhang, Mingyang, Bendersky, Michael and Najork, Marc
by Guo, Demi , Rush, Alexander and Kim, YoonThe approach learns a task-specific “diff” vector that extends the original pretrained parameters. As the number of tasks increases, diff pruning remains parameter-efficient, as it requires storing only a small diff vector for each task.
by Cui, Li , Yang, Deqing , Yu, Jiaxin , Hu, Chengwei , Cheng, Jiayang , Yi, Jingjie and Xiao, YanghuaTo fully utilize memorized samples, in this paper, we employ relation prototype to extract useful information of each relation.
by He, Ruidan , Liu, Linlin , Ye, Hai , Tan, Qingyu , Ding, Bosheng , Cheng, Liying , Low, Jiawei , Bing, Lidong et al.we first show that adapter-based tuning better mitigates forgetting issues than fine-tuning since it yields representations with less deviation from those generated by the initial PrLM. Effectiveness: it tendsto outperform fine-tuning on both low-resource and cross-lingual tasks; 2 it demonstrates higher stability under different learning rates compared to fine-tuning.
by Kanwatchara, Kasidis , Horsuwan, Thanapapas , Lertvittayakumjorn, Piyawat , Kijsirikul, Boonserm and Vateekul, PeeraponRational LAMOL enhances LAMOL, a recent LL model, by applying critical freezing guided by human rationales. When the human rationales are not available, we propose exploiting unsupervised generated rationales as substitutions.Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution,
by Garcia, Xavier , Constant, Noah , Parikh, Ankur and Firat, OrhanIntroducing the catastrophic forgetting problem in incremental multi-language translation, and utilizing a vocabulary substitution manner to alleviate the above problem.
by Huang, Yufan , Zhang, Yanzhe , Chen, Jiaao , Wang, Xuezhi and Yang, DiyiProposing a regularization-based method for continual text classification, introducing the next sentence prediction and task id prediction as auxiliary tasks.
by Xia, Congying , Yin, Wenpeng , Feng, Yihao and Yu, PhilipProposing a new setting and respective benchmark for few-shot incremental text classification, modeling continual text classification with text entailment.
by Hua, Ting , Shen, Yilin , Zhao, Changsheng , Hsu, Yen-Chang and Jin, HongxiaInspired by EWC and proposing a hyperparameter-free (Fisher information-based) sampling method for memory replay.
by Vijayaraghavan, Prashanth and Roy, DebProposing a rehearsal-based method, i.e.,Domain-Representative Episodic Memory Replay (DR-EMR), for lifelong event representation with embedding alignment and external social commonsense knowledge.
by Qingbin Liu, Xiaoyan Yu, Shizhu He, Kang Liu and Jun ZhaoWe propose the lifelong intent detection task to handle continually emerging user intents. And, we propose multistrategy rebalancing to address multiple adverse effects caused by the data imbalance problem.
by Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu and Xiangzhan YuWe propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks. Specifically, we introduce a Pretraining Simulation mechanism to recall the knowledge from pretraining tasks without data, and an Objective Shifting mechanism to focus the learning on downstream tasks gradually.
by Zhaojiang Lin, Andrea Madotto and Pascale FungProposing an adapter-based method for continual learning in text generation. One of the insights is a frozen PLM can be well-applied in continual learning.
by Arumae, Kristjan , Sun, Qing and Bhatia, ParminderWe find that elastic weight consolidation provides best overall scores yielding only a 0.33% drop in performance across seven generic tasks while remaining competitive in bio-medical tasks.
by Jin, Xisen , Du, Junyi , Sadhu, Arka , Nevatia, Ram and Ren, XiangA novel continual learning setting and a new benchmark for continual caption generation, evaluated with exiting rehearsal-based methods
by Cao, Pengfei , Chen, Yubo , Zhao, Jun and Wang, TaifengProposing a hybrid continual learning method for event detection, combining experience replay and Knowledge Distillation, focusing on (1) semantic ambiguity in NLP and (2) data imbalance between memory and current task.
by Dai, Zehui , Peng, Cheng , Chen, Huajie and Ding, YadongUtilizing BERT for sentence and category encoding, preserving category encoding to prevent catastrophic forgetting.
by Wang, Zirui , Mehta, Sanket Vaibhav , Poczos, Barnabas and Carbonell, JaimeA meta learning-enhanced version of MbPA (NeurIPS19), sharing the continual setting as well. Figure 1 is interesting.
by Chuang, Yung-Sung , Su, Shang-Yu and Chen, Yun-NungProposing a Knowledge Distillation-enhanced Method LLL based on LAMOL (ICLR 2020) model for continual learning, evaluated on text generation and text classification.
by Sun, Jingyuan , Wang, Shaonan , Zhang, Jiajun and Zong, ChengqingProposing a distill and replay method (DnR) which follows the setting of LAMOL. As a distillation-based method, DnR also shows the ability in incrementally compressing the model size while still outperforming most of the baselines.
by Sun, Yu, Wang, Shuohuan, Li, Yukun, Feng, Shikun, Tian, Hao, Wu, Hua and Wang, HaifengIn order to extract the lexical, syntactic and semantic information from training corpora, we propose a continual pre-training framework named ERNIE 2.0 which incrementally builds pre-training tasks and then learn pre-trained models on these constructed tasks via continual multi-task learning.
by Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong and Dani YogatamaMbPA++. This paper proposes the use of memory (a fixed memory network) in life-long learning to prevent catastrophic forgetting by means of experience replay and local adaptation.
by Terry Yue Zhuo, Zhuang Li, Yujin Huang, Yuan-Fang Li, Weiqing Wang, Gholamreza Haffari and Fatemeh Shiri
by Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith et al.
by Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith and Douglas C. Schmidt
by Jiawei Zhang
by Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Cote, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner et al.
by Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot et al.
by Albert Webson and Ellie Pavlick
by Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan and Jimmy Ba
by Taylor Sorensen, Joshua Robinson, Christopher Michael Rytting, Alexander Glenn Shaw, Kyle Jeffrey Rogers, Alexia Pauline Delorey, Mahmoud Khalil, Nancy Fulda et al.
by Hila Gonen, Srini Iyer, Terra Blevins, Noah A. Smith and Luke Zettlemoyer
by Robert L. Logan IV, Ivana Balazevic, Eric Wallace, Fabio Petroni, Sameer Singh and Sebastian Riedel
by Hui Wu and Xiaodong Shi
by Fredrik Carlsson, Joey "Ohman, Fangyu Liu, Severine Verlinden, Joakim Nivre and Magnus Sahlgren
by Zhixing Tan, Xiangwen Zhang, Shuo Wang and Yang Liu
by Sewon Min, Mike Lewis, Hannaneh Hajishirzi and Luke Zettlemoyer
by Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou' and Daniel Cer
by Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen et al.Meta-Adapters: Parameter Efficient Few-shot Fine-tuning through Meta-Learning,
by Trapit Bansal, Salaheddin Alzubi, Tong Wang, Jay-Yoon Lee and Andrew McCallum
by Shengding Hu, Zhen Zhang, Ning Ding, Yadao Wang, Yasheng Wang, Zhiyuan Liu and Maosong SunOntology-enhanced Prompt-tuning for Few-shot Learning,
by Hongbin Ye, Ningyu Zhang, Shumin Deng, Xiang Chen, Hui Chen, Feiyu Xiong, Xi Chen and Huajun Chen
by Xuandong Zhao, Siqi Ouyang, Zhiguo Yu, Ming Wu and Lei Li
by Denny Zhou, Nathanael Sch"arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet et al.(1) 两阶段的prompt，第一阶段问题分解（通过in-context learning实现，context中包含了其他问题的分解示例），对于每个问题，分解出回答该问题需要先回答什么子问题；(2) 在第二阶段中，从后往前依次解决子问题，同样通过in-context learing得到，每次LLM的回答会参与组成下一个问题的prompt。
by Ye, Xi and Durrett, Greg
by Simran Arora, Avanika Narayan, Mayee F. Chen, Laurel J. Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala et al.
by Boxi Cao, Hongyu Lin, Xianpei Han, Fangchao Liu and Le Sun
by Daniel Khashabi, Chitta Baral, Yejin Choi and Hannaneh Hajishirzi
by Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman, Yulia Tsvetkov and Luke Zettlemoyer
by Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang et al.
by Xiangyu Peng, Chen Xing, Prafulla Kumar Choubey, Chien-Sheng Wu and Caiming Xiong
by Rakesh Chada and Pradeep Natarajan
by Brian Lester, Rami Al-Rfou and Noah Constant
by Xiang Lisa Li and Percy LiangPrompt Programming for Large Language Models: Beyond the Few-Shot Paradigm,
by Laria Reynolds and Kyle McDonell
by OpenAI
by OpenAI
by Jianing Wang, Wenkang Huang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Xiang Li and Ming Gao
by Dou Hu, Xiaolong Hou, Xiyang Du, Mengyuan Zhou, Lianxin Jiang, Yang Mo and Xiaofeng Shi
by Yu Meng, Jiaxin Huang, Yu Zhang and Jiawei Han
by Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang and Luo Si
by Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou et al.
by Radford, Alec, Narasimhan, Karthik, Salimans, Tim, Sutskever, Ilya and others
by Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron et al.
by Driess, Danny, Xia, Fei, Sajjadi, Mehdi SM, Lynch, Corey, Chowdhery, Aakanksha, Ichter, Brian, Wahid, Ayzaan, Tompson, Jonathan et al.
by Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee and Chunyuan Li
by Wu, Chenfei, Yin, Shengming, Qi, Weizhen, Wang, Xiaodong, Tang, Zecheng and Duan, Nan
by Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh et al.
by Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon et al.Multimodal Pretraining from Monolingual to Multilingual,
by Liang Zhang, Ludan Ruan, Anwen Hu and Qin Jin
by Guyue Hu, Bin He and Hanwang Zhang
by Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li and Mohamed Elhoseiny
by Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu et al.
by Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji et al.
by Hsiu-Yu Yang and Carina Silberer
by Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer et al.
by Xuran Pan, Tianzhu Ye, Dongchen Han, Shiji Song and Gao Huang
by Xin Wang, Yasheng Wang, Pingyi Zhou, Fei Mi, Meng Xiao, Yadao Wang, Li Li, Xiao Liu et al.Less Is More: ClipBERT for Video-and-Language Learning via Sparse Sampling,
by Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal and Jingjing Liu
by Ronghang Hu and Amanpreet SinghPre-training Graph Transformer with Multimodal Side Information for Recommendation,
by Yong Liu, Susen Yang, Chenyi Lei, Guoxin Wang, Haihong Tang, Juyong Zhang, Aixin Sun and Chunyan Miao
by Huaishao Luo, Lei Ji, Botian Shi, Haoyang Huang, Nan Duan, Tianrui Li, Xilin Chen and Ming Zhou
by Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng and Jingjing Liu
by Hao Tan and Mohit Bansal
by Wasifur Rahman, Md. Kamrul Hasan, Sangwu Lee, AmirAli Bagher Zadeh, Chengfeng Mao, Louis-Philippe Morency and Mohammed E. Hoque
by Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei and Jifeng Dai
by Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh and Kai-Wei Chang
by Jiasen Lu, Dhruv Batra, Devi Parikh and Stefan Lee
by Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy and Cordelia Schmid
by Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li and Kai-Wei Chang
by Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang et al.
by Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber and Lijuan Wang
by Dustin Tran, Jeremiah Z. Liu, Michael W. Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang et al.
by Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran and Mario Lucic
by Archit Karandikar, Nicholas Cain, Dustin Tran, Balaji Lakshminarayanan, Jonathon Shlens, Michael C. Mozer and Becca Roelofs
by Swaroop Mishra, Bhavdeep Singh Sachdeva and Chitta Baral
by Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan and Dawn Song
by Yuxian Gu, Jiaxin Wen, Hao Sun, Yi Song, Pei Ke, Chujie Zheng, Zheng Zhang, Jianzhu Yao et al.
by Praveen Venkateswaran, Evelyn Duesterwald and Vatche Isahagian
by Young-Jun Lee, Chae-Gyun Lim and Ho-Jin Choi
by Tom Young, Frank Xing, Vlad Pandelea, Jinjie Ni and Erik Cambria
by Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett, Lars Liden, Elnaz Nouri, Zhou Yu, Bill Dolan et al.
by Sagi Shaier, Lawrence Hunter and Katharina Kann
by Chia-Hsuan Lee, Hao Cheng and Mari Ostendorf
by Andrea Madotto, Zhaojiang Lin, Genta Indra Winata and Pascale Fung
by Derek Chen, Howard Chen, Yi Yang, Alexander Lin and Zhou Yu
by Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko and Jungyun Seo
by Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Vinay Adiga and Erik CambriaSlot Self-Attentive Dialogue State Tracking,
by Fanghua Ye, Jarana Manotumruksa, Qiang Zhang, Shenghui Li and Emine YilmazPretraining the Noisy Channel Model for Task-Oriented Dialogue,
by Qi Liu, Lei Yu, Laura Rimell and Phil Blunsom
by Yunyi Yang, Yunhao Li and Xiaojun Quan
by DongHoon Ham, Jeong-Gwan Lee, Youngsoo Jang and Kee-Eung Kim
by Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz and Richard Socher
by Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang and Jiawei Zhang
by Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin and Ji-Rong Wen
by Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang and Qing Li
by Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong and Qing HeAre Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation,
by Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui and Quoc Viet Hung NguyenDisentangled Representations Learning for Multi-Target Cross-Domain Recommendation,
by Guo, Xiaobo, Li, Shaoshuai, Guo, Naicheng, Cao, Jiangxia, Liu, Xiaolei, Ma, Qiongxu, Gan, Runsheng and Zhao, Yunan
by Xin Xin, Tiago Pimentel, Alexandros Karatzoglou, Pengjie Ren, Konstantina Christakopoulou and Zhaochun Ren
by Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke and Tat-Seng Chua
by Yong Liu, Susen Yang, Chenyi Lei, Guoxin Wang, Haihong Tang, Juyong Zhang, Aixin Sun and Chunyan Miao
by Tong Yu, Yilin Shen and Hongxia Jin
by Amir Pouran Ben Veyseh and Thien Huu Nguyen
by Minh Van Nguyen, Bonan Min, Franck Dernoncourt and Thien Nguyen
by Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji et al.Augmenting Open-Domain Event Detection with Synthetic Data from GPT-2,
by Amir Pouran Ben Veyseh, Minh Van Nguyen, Bonan Min and Thien Huu Nguyen
by Rongzhi Zhang, Yue Yu and Chao Zhang
by Sen Yang, Dawei Feng, Linbo Qiao, Zhigang Kan and Dongsheng Li
Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation,
by Jinglong Gao, Xiao Ding, Bing Qin and Ting Liu
by Minh Van Nguyen, Bonan Min, Franck Dernoncourt and Thien Nguyen
by Hieu Man, Nghia Trung Ngo, Linh Ngo Van and Thien Huu Nguyen
by Viet Dac Lai, Hieu Man, Linh Ngo Van, Franck Dernoncourt and Thien Nguyen
by Shirong Shen, Heng Zhou, Tongtong Wu and Guilin Qi
by Xiyang Zhang, Muhao Chen and Jonathan May
by Haoyu Wang, Muhao Chen, Hongming Zhang and Dan Roth
by Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Zihao Wu, Lin Zhao, Wei Liu, Ninghao Liu et al.
by Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael W. Dusenberry, Jasper Snoek, Balaji Lakshminarayanan and Dustin Tran
by Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee and Woo-Myoung Park
by Rongzhi Zhang, Yue Yu and Chao Zhang
by Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing, Shafiq R. Joty and Boyang Li
by Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu and Michael Zeng
by Xinliang Frederick Zhang, Carter Wood Blum, Temma Choji, Shalin Shah and Alakananda Vempala
by Yuzhao Heng, Chunyuan Deng, Yitong Li, Yue Yu, Yinghao Li, Rongzhi Zhang and Chao Zhang
by Yinghao Li, Rampi Ramprasad and Chao Zhang
by Ruirui Chen, Chengwei Qin, Weifeng Jiang and Dongkyu Choi
by Anonymous Submission
by Haojie Zhang and Yimeng Zhuang
by Junjie Ye, Nuo Xu, Yikun Wang, Jie Zhou, Qi Zhang, Tao Gui and Xuanjing Huang
by Monica Munnangi, Sergey Feldman, Byron C Wallace, Silvio Amir, Tom Hope and Aakanksha Naik
by Loka Li, Guangyi Chen, Yusheng Su, Zhenhao Chen, Yixuan Zhang, Eric Xing and Kun Zhang
by Guozheng Li, Wenjun Ke, Peng Wang, Zijie Xu, Ke Ji, Jiajun Liu, Ziyu Shang and Qiqing Luo
by Jingping Liu, Mingchuan Zhang, Weichen Li, Chao Wang, Shuang Li, Haiyun Jiang, Sihang Jiang, Yanghua Xiao et al.
by Xuemei Tang, Jun Wang and Qi Su
by Yilmazcan Ozyurt, Stefan Feuerriegel and Ce Zhang
by Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu et al.
by Terra Blevins, Hila Gonen and Luke Zettlemoyer
by Siyu Yuan, Deqing Yang, Jinxi Liu, Shuyu Tian, Jiaqing Liang, Yanghua Xiao and Rui Xie
by Somin Wadhwa, Silvio Amir and Byron C. Wallace
by Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu and Xiang Wan
by Jiawei Chen, Yaojie Lu, Hongyu Lin, Jie Lou, Wei Jia, Dai Dai, Hua Wu, Boxi Cao et al.
by Chenxi Whitehouse, Clara Vania, Alham Fikri Aji, Christos Christodoulopoulos and Andrea Pierleoni
by Kai Zhang, Bernal Jimenez Gutierrez and Yu Su
by Guozheng Li, Peng Wang and Wenjun Ke
by Keming Lu, Xiaoman Pan, Kaiqiang Song, Hongming Zhang, Dong Yu and Jianshu Chen
by Alexander Brinkmann, Roee Shraga and Christian Bizer
by Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li and Sadao KurohashiText2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text,
by Nandana Mihindukulasooriya, Sanju Tiwari, Carlos F. Enguix and Kusum Lata- <img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics:_{EMNLP}
2023,_Singapore,_December_6--10,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> PIVOINE: Instruction Tuning for Open-world Entity Profiling,
by Keming Lu, Xiaoman Pan, Kaiqiang Song, Hongming Zhang, Dong Yu and Jianshu Chen
by Andrea Papaluca, Daniel Krefl, Sergio Mendez Rodriguez, Artem Lensky and Hanna Suominen
by Yizhu Jiao, Ming Zhong, Sha Li, Ruining Zhao, Siru Ouyang, Heng Ji and Jiawei Han
by Andrea Papaluca, Daniel Krefl, Sergio Mendez Rodriguez, Artem Lenskiy and Hanna Suominen
by Tingyu Xie, Qi Li, Jian Zhang, Yan Zhang, Zuozhu Liu and Hongwei Wang
by Sneha Singhania, Simon Razniewski and Gerhard WeikumZero-shot Triplet Extraction by Template Infilling,
by Bosung Kim, Hayate Iso, Nikita Bhutani, Estevam Hruschka, Ndapa Nakashole and Tom M. MitchellLLM Instruction-Example Adaptive Prompting (LEAP) Framework for Clinical Relation Extraction,
by Anonymous Submission
by Yubo Ma, Yixin Cao, Yong Hong and Aixin Sun
by Xilai Ma, Jing Li and Min Zhang
by Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao and Hoifung Poon
by Chaoxu Pang, Yixuan Cao, Qiang Ding and Ping Luo
by Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim and David A. Sontag
by Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun and Yu Su
by Jiacheng Li, Yannis Katsis, Tyler Baldwin, Ho-Cheol Kim, Andrew Bartko, Julian J. McAuley and Chun-Nan Hsu
by Kesong Liu, Jianhui Jiang and Feifei Lyu
by Daixuan Cheng, Shaohan Huang, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Furu Wei, Denvy Deng and Qi Zhang
by Dou Hu, Xiaolong Hou, Xiyang Du, Mengyuan Zhou, Lianxin Jiang, Yang Mo and Xiaofeng Shi
by Jiajie Zhang, Shulin Cao, Linmei Hu, Ling Feng, Lei Hou and Juanzi Li
by Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W. Huang, Jie Fu et al.
by Yuanyuan Liang, Keren Tan, Tingyu Xie, Wenbiao Tao, Siyuan Wang, Yunshi Lan and Weining Qian
by Zhijie Nie, Richong Zhang, Zhongyuan Wang and Xudong Liu
by Reham Omar, Omij Mangukiya, Panos Kalnis and Essam Mansour
by Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen and Guilin Qi
by Dhruv Agarwal, Rajarshi Das, Sopan Khosla and Rashmi Gangadharaiah
by Zhou Yu, Xuecheng Ouyang, Zhenwei Shao, Meng Wang and Jun Yu
by Chaojie Wang, Yishi Xu, Zhong Peng, Chenxi Zhang, Bo Chen, Xinrun Wang, Lei Feng and Bo An
by Chuanyuan Tan, Yuehe Chen, Wenbiao Shao and Wenliang Chen
by Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su and Wenhu Chen
by Yunlong Chen, Yaming Zhang, Jianfei Yu, Li Yang and Rui Xia
by Xiao Liu, Shiyu Zhao, Kai Su, Yukuo Cen, Jiezhong Qiu, Mengdi Zhang, Wei Wu, Yuxiao Dong et al.
by Apoorv Saxena, Adrian Kochsiek and Rainer Gemulla
by Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir R. Radev, Noah A. Smith et al.
by Jinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeon-Jin Park, Ji-Hoon Kim, Jisu Jeong, Kyung-Min Kim and Hyunwoo J. Kim
by Shengzhe Xu, Christo Kurisummoottil Thomas, Omar Hashash, Nikhil Muralidhar, Walid Saad and Naren Ramakrishnan
by Yangning Li, Shirong Ma, Xiaobin Wang, Shen Huang, Chengyue Jiang, Haitao Zheng, Pengjun Xie, Fei Huang et al.Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models,
by Kung, Tiffany H, Cheatham, Morgan, Medenilla, Arielle, Sillos, Czarina, De Leon, Lorie, Elepa~no, Camille, Madriaga, Maria, Aggabao, Rimel et al.
by Mbakwe, Amarachi B, Lourentzou, Ismini, Celi, Leo Anthony, Mechanic, Oren J and Dagan, Alon
by Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg et al.
by Liying Cheng, Xingxuan Li and Lidong Bing
by Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou, Zhuo Han, Songyang Zhang, Kai Chen, Zongwen Shen et al.
by Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher R'e, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters et al.
by Xuan Xiao, Jiahang Liu, Zhipeng Wang, Yanmin Zhou, Yong Qi, Qian Cheng, Bin He and Shuo JiangTowards the Generation of Musical Explanations with GPT-3,
by Stephen James Krol, Maria Teresa Llano and Jon McCormack
by Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis and He HeThis paper proposes in-context tuning, which recasts task adaptation and prediction as a simple sequence prediction problem: to form the input sequence, concatenate the task instruction, labeled in-context examples, and the target input to predict; to meta train the model to learn from in-context examples, finetune a PLM to predict the target label given the input sequence on a collection of tasks (very similar to MetaICL). On LAMA and BinaryClfs, the proposed method outperforms MAML.
by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh HajishirziMetaICL proposes a supervised meta-training framework to enable LMs to more effectively learn a new task in context. In MetaICL, each meta-training example includes several training examples from one task that will be presented together as a single sequence to the LM, and the prediction of the final example is used to calculate the loss.
by J. D. Zamfirescu-Pereira, Bjoern Hartmann and Qian Yang
by Hang Hua, Xingjian Li, Dejing Dou, Cheng-Zhong Xu and Jiebo Luo
by Peifeng Wang, Filip Ilievski, Muhao Chen and Xiang Ren
by Mehran Kazemi, Sid Mittal and Deepak RamachandranLarge Language Models Struggle to Learn Long-Tail Knowledge,
by Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace and Colin Raffel
by Roi Cohen, Mor Geva, Jonathan Berant and Amir Globerson
by Pouya Pezeshkpour
by Linyao Yang, Hongyang Chen, Zhao Li, Xiao Ding and Xindong Wu
by Yuan He, Jiaoyan Chen, Ernesto Jim'enez-Ruiz, Hang Dong and Ian Horrocks
by Shibo Hao, Bowen Tan, Kaiwen Tang, Bin Ni, Xiyan Shao, Hengzhe Zhang, Eric P. Xing and Zhiting Hu
by Pengcheng Jiang, Shivam Agarwal, Bowen Jin, Xuan Wang, Jimeng Sun and Jiawei Han
by Vishwas Mruthyunjaya, Pouya Pezeshkpour, Estevam Hruschka and Nikita Bhutani
by Paul Youssef, Osman Alperen Koras, Meijie Li, J"org Schl"otterer and Christin Seifert
by Ruilin Zhao, Feng Zhao, Guandong Xu, Sixiao Zhang and Hai Jin
by Victor Milewski, Miryam de Lhoneux and Marie-Francine Moens
by Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona T. Diab and Marjan GhazvininejadTime-Aware Language Models as Temporal Knowledge Bases,
by Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein and William W. Cohen
by Dimitrios Alivanistos, Selene Baez Santamar'\ia, Michael Cochez, Jan-Christoph Kalo, Emile van Krieken and Thiviyan Thanapalasingam
by Benjamin Heinzerling and Kentaro Inui
by Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue and Jin Xu
by Mujeen Sung, Jinhyuk Lee, Sean S. Yi, Minji Jeon, Sungdong Kim and Jaewoo Kang
by Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace and Sameer Singh
by Fabio Petroni, Tim Rockt"aschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu and Alexander H. Miller
by Shi-Qi Yan, Jia-Chen Gu, Yun Zhu and Zhen-Hua Ling
by Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown and Yoav Shoham
by Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee and Chunyuan Li
by Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer and Wen-tau Yih
by Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu et al.
by Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jingyuan Wang, Jian-Yun Nie and Ji-Rong Wen
by Chao Feng, Xinyu Zhang and Zichu Fei
by Huayang Li, Yixuan Su, Deng Cai, Yan Wang and Lemao Liu
by Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer et al.Atlas: Few-shot learning with retrieval augmented language models,
by Izacard, Gautier, Lewis, Patrick, Lomeli, Maria, Hosseini, Lucas, Petroni, Fabio, Schick, Timo, Dwivedi-Yu, Jane, Joulin, Armand et al.
by Zexuan Zhong, Tao Lei and Danqi Chen
by Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau et al.
by Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat and Ming-Wei Chang
by Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K"uttler, Mike Lewis et al.
by Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi and Yejin ChoiMeasuring and Improving Consistency in Pretrained Language Models,
by Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard H. Hovy, Hinrich Sch"utze and Yoav Goldberg
by Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff and Dan Roth
by Swarnadeep Saha, Peter Hase, Nazneen Rajani and Mohit Bansal
by Bhargavi Paranjape, Julian Michael, Marjan Ghazvininejad, Hannaneh Hajishirzi and Luke Zettlemoyer
by Gao, Jiahui, Pi, Renjie, Yong, LIN, Xu, Hang, Ye, Jiacheng, Wu, Zhiyong, ZHANG, WEIZHONG, Liang, Xiaodan et al.
by Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu and Lingpeng Kong
by Yu Meng, Jiaxin Huang, Yu Zhang and Jiawei Han
by Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng and Minlie Huang
by Shengxiang Hu, Guobing Zou, Song Yang, Yanglan Gan, Bofeng Zhang and Yixin Chen
by Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin et al.
by Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu and Yongfeng Zhang
by Ziwei Zhang, Haoyang Li, Zeyang Zhang, Yijian Qin, Xin Wang and Wenwu Zhu
by Lanning Wei, Zhiqiang He, Huan Zhao and Quanming Yao
by Zheng Chen, Ziyan Jiang, Fan Yang, Eunah Cho, Xing Fan, Xiaojiang Huang, Yanbin Lu and Aram Galstyan
by Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu and Jun Zhao
by Pratyush Maini, Michael Curtis Mozer, Hanie Sedghi, Zachary Chase Lipton, J. Zico Kolter and Chiyuan Zhang
by Mor Geva, Roei Schuster, Jonathan Berant and Omer Levy
by Xuhui Jiang, Yinghan Shen, Zhichao Shi, Chengjin Xu, Wei Li, Zixuan Li, Jian Guo, Huawei Shen et al.
by Linyao Yang, Hongyang Chen, Xiao Wang, Jing Yang, Fei-Yue Wang and Han Liu
by Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Shikhar Vashishth, Sriram Ganapathy, Abhishek Bapna, Prateek Jain et al.
by Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade and Jacob Solawetz
by Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim, Sheikh Shafayat and Minjoon Seo
by Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran and Yulia Tsvetkov
by Linyao Yang, Hongyang Chen, Xiao Wang, Jing Yang, Fei-Yue Wang and Han Liu
by Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro and Pengxiang Cheng
by Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang and Dacheng Tao
by Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel and Mohit Bansal
by Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal and Tianlong Chen
by Nico Daheim, Thomas Möllenhoff, Edoardo Maria Ponti, Iryna Gurevych and Mohammad Emtiyaz Khan
by Hongyi Wang, Felipe Maia Polo, Yuekai Sun, Souvik Kundu, Eric P. Xing and Mikhail Yurochkin
by Tao Feng, Zifeng Wang and Jimeng Sun
by Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Chengfei Lv and Huajun Chen
by Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu and Zhaoran Wang
All in One: Multi-Task Prompting for Graph Neural Networks,
by Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu and Jihong Guan
by Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V. Chawla and Panpan Xu
by Xiangguo Sun, Jiawen Zhang, Xixi Wu, Hong Cheng, Yun Xiong and Jia Li
by Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji and Jiawei Han
by Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang and Xin Wang
- <img src=https://img.shields.io/badge/{IEEE/CVF}_Conference_on_Computer_Vision_and_Pattern_Recognition,
{CVPR}_2023,_Vancouver,_BC,_Canada,_June_17--24,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visual Atoms: Pre-Training Vision Transformers with Sinusoidal Waves,
by Sora Takashima, Ryo Hayamizu, Nakamasa Inoue, Hirokatsu Kataoka and Rio Yokota
by Fali Wang, Runxue Bao, Suhang Wang, Wenchao Yu, Yanchi Liu, Wei Cheng and Haifeng ChenGive Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling,
by Yang, Linyao, Chen, Hongyang, Li, Zhao, Ding, Xiao and Wu, Xindong
by Jonathan Larson, Steven TruittGraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph,
by Xin Li, Dongze Lian, Zhihe Lu, Jiawang Bai, Zhibo Chen and Xinchao Wang
by Yubin Wang, Xinyang Jiang, De Cheng, Dongsheng Li and Cairong Zhao
by Yansong Ning and Hao Liu
by Qiyuan He, Yizhong Wang and Wenya WangCan Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey,
by Garima Agrawal, Tharindu Kumarage, Zeyad Alghami and Huan Liu
by Xinyan Guan, Yanjiang Liu, Hongyu Lin, Yaojie Lu, Ben He, Xianpei Han and Le Sun
by Yihao Li, Ru Zhang, Jianyi Liu and Gongshen Liu
by Xin Zhang, Tianjie Ju, Huijia Liang, Ying Fu and Qin Zhang
by Rudy Agovic
by Neo4j
by Yichi Zhang, Zhuo Chen, Wen Zhang and Huajun ChenLarge Language Models and Knowledge Graphs: Opportunities and Challenges,
by Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jiaoyan Chen, Stefan Dietze, Hajira Jabeen, Janna Omeliyanenko et al.
by Blerta Veseli, Simon Razniewski, Jan-Christoph Kalo and Gerhard Weikum
by Justin D. Weisz, Michael J. Muller, Jessica He and Stephanie Houde
by Zijian Zhang, Zhou Zhao and Zhijie Lin
by David Vilares, Michalina Strzyz, Anders S\ogaard and Carlos G'omez-Rodr'\iguez
by Jiabo Huang, Qi Dong, Shaogang Gong and Xiatian ZhuHellaSwag: Can a Machine Really Finish Your Sentence?,
by Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi and Yejin ChoiLearning to detect unseen object classes by between-class attribute transfer,
by Christoph H. Lampert, Hannes Nickisch and Stefan Harmeling
-
Awesome-ChatGPT,
ChatGPT资料汇总学习，持续更新...... -
Awesome ChatGPT Prompts,
In this repository, you will find a variety of prompts that can be used with ChatGPT. -
ChatRWKV,
ChatRWKV is like ChatGPT but powered by my RWKV (100% RNN) language model, which is the only RNN (as of now) that can match transformers in quality and scaling, while being faster and saves VRAM. Training sponsored by Stability EleutherAI. -
ChatGPT-Hub,
ChatGPT资源汇总 -
PaLM-rlhf-pytorch,
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. -
BAAI-WuDao/Data,
“悟道”项目构建了高质量的数据集，用于支撑大模型的训练和测评工作，本仓库提供所有开源数据集的链接。 -
Colossal-AI,
Colossal-AI provides a collection of parallel components for you. We aim to support you to write your distributed deep learning models just like how you write your model on your laptop. We provide user-friendly tools to kickstart distributed training and inference in a few lines.
-
Exploring Prompt Injection Attacks,
by Jose Selvi
Prompt Injection is a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning. -
ChatGPT发展历程、原理、技术架构详解和产业未来,
by 陈巍
本文将介绍ChatGPT的特点、功能、技术架构、局限、产业应用、投资机会和未来。作者本人曾担任华为系自然语言处理（ NLP ）企业的首席科学家。
-
How does GPT Obtain its Ability?,
by Yao Fu
Tracing emergent abilities of language models to their sources. -
Open source solution replicates ChatGPT training process,
Colossal-AI, as one of the hottest open-source solutions for large AI models, presents an open-source low-cost ChatGPT equivalent implementation process.
- CPM-Bee,
CPM-Bee是一个开源的双语预训练语言模型，参数量为10B，拥有十余种原生能力和强大的通用语言能力，并支持结构化输入和输出。
-
Awesome-ChatGPT,
ChatGPT资料汇总学习，持续更新...... -
Awesome ChatGPT Prompts,
In this repository, you will find a variety of prompts that can be used with ChatGPT. -
ChatRWKV,
ChatRWKV is like ChatGPT but powered by my RWKV (100% RNN) language model, which is the only RNN (as of now) that can match transformers in quality and scaling, while being faster and saves VRAM. Training sponsored by Stability EleutherAI. -
ChatGPT-Hub,
ChatGPT资源汇总 -
PaLM-rlhf-pytorch,
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. -
BAAI-WuDao/Data,
“悟道”项目构建了高质量的数据集，用于支撑大模型的训练和测评工作，本仓库提供所有开源数据集的链接。 -
Colossal-AI,
Colossal-AI provides a collection of parallel components for you. We aim to support you to write your distributed deep learning models just like how you write your model on your laptop. We provide user-friendly tools to kickstart distributed training and inference in a few lines.
-
Exploring Prompt Injection Attacks,
by Jose Selvi
Prompt Injection is a new vulnerability that is affecting some AI/ML models and, in particular, certain types of language models using prompt-based learning. -
ChatGPT发展历程、原理、技术架构详解和产业未来,
by 陈巍
本文将介绍ChatGPT的特点、功能、技术架构、局限、产业应用、投资机会和未来。作者本人曾担任华为系自然语言处理（ NLP ）企业的首席科学家。
-
How does GPT Obtain its Ability?,
by Yao Fu
Tracing emergent abilities of language models to their sources. -
Open source solution replicates ChatGPT training process,
Colossal-AI, as one of the hottest open-source solutions for large AI models, presents an open-source low-cost ChatGPT equivalent implementation process.
- CPM-Bee,
CPM-Bee是一个开源的双语预训练语言模型，参数量为10B，拥有十余种原生能力和强大的通用语言能力，并支持结构化输入和输出。
Knowledge Science and Engineering Lab is recruiting researchers! You are welcome to apply for the following positions:
- Research Assistant: Bachelor degree or above, proficient in Python/Java, familiar with machine learning espicially deep learning models.
- Postdoctoral Fellow: Doctoral research in Artificial Intelligence, published at least 3 high-quality papers.
- Lecturer, Associate Professor and Professor
If you are interested in our research and meet the above requirements, feel free to contact Prof. Guilin Qi.
知识科学与工程实验室正在招聘科研人员！欢迎申请以下岗位：
- 科研助理：本科学历以上，精通Python/Java，熟悉机器学习，特别是深度学习模型。
- 博士后：博士研究人工智能相关方向，发表至少3篇高水平论文。
- 讲师、副教授、教授等教职
如果您对我们的研究工作感兴趣并满足以上要求，欢迎您与漆桂林教授联系。