矣晓沅：清华古典诗歌自动生成系统“九歌”的算法
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">导读：</span></strong><span style="color: black;">本文将介绍清华大学THUNLP实验自研的<span style="color: black;">拥有</span>文学表现力的中文古典诗歌自动生成系统“九歌”的算法，<span style="color: black;">重点</span><span style="color: black;">包含</span>以下几方面内容：</span></span></p><span style="color: black;"><span style="color: black;">自动作诗缘起</span></span><span style="color: black;"><span style="color: black;">九歌的模型算法</span></span><span style="color: black;"><span style="color: black;">九歌系统介绍</span></span><span style="color: black;"><span style="color: black;">自动作诗与知识图谱</span></span>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">--</span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">01</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">自动作诗缘起</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">1. 诗歌自动写作</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">① 何为诗歌自动写作</span></strong></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/78d61dd81c4a41a48fd09933b0220b20~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=Zk4CF789puu4lNpM0uLIfw36zec%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">诗歌自动写作，是用户给定某种形式的输入，<span style="color: black;">例如</span>关键词、语句段落、<span style="color: black;">照片</span>等，由生成模型依据输入生成一首完整的诗歌，所生成的诗歌，既要满足<span style="color: black;">必定</span>的形式<span style="color: black;">需求</span>，<span style="color: black;">例如</span>长度、平仄，又要满足<span style="color: black;">必定</span>的语义<span style="color: black;">需求</span>，<span style="color: black;">例如</span>语句通顺，连贯一致等。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">② 诗歌自动写作的<span style="color: black;">科研</span>价值</span></strong></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">初心：探索<span style="color: black;">设备</span>智能</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/a9d06eb45d984fce86e02a379783db51~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=Gi0q4H1%2FC23gCJIQNtZynDzCS%2BU%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">这个问题<span style="color: black;">拥有</span>非常丰富的<span style="color: black;">科研</span>价值，而<span style="color: black;">咱们</span>做它的初心是探索<span style="color: black;">设备</span>智能。人工智能之父Alan Turing就曾对人工智能写诗有过描述和设想。在史<span style="color: black;">叫作</span>图灵-杰斐逊辩论的讨论和1950年其关于图灵测试的论文中都有所<span style="color: black;">表现</span>。图灵认为人工智能是有可能写出有韵律的诗歌并且能对诗中的格律意象有<span style="color: black;">必定</span>的理解。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p26-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/8538e222b4b140cbb3ff18a5e4c3be47~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=BPv9Nv8iAG1np8%2BTJZZlyOELTnw%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">人工智能的经典任务是围棋，它考验的是<span style="color: black;">规律</span>推理能力，而诗歌自动创作考验的是创造能力。假设<span style="color: black;">全部</span>汉字集中有约一万个字，<span style="color: black;">那样</span>所有可能的七律的数量，远大于围棋棋盘可能的盘面数和宇宙中所有的原子数。我国著名的科幻小说家刘慈欣在短篇小说《诗云》一书中构想了一个高级智能文明，能够把所有诗歌的可能<span style="color: black;">状况</span><span style="color: black;">所有</span>生成并<span style="color: black;">保留</span>下来，但<span style="color: black;">无</span>办法去找出其中真正有价值的好诗歌。<span style="color: black;">怎样</span>从海量可能性中的把前人<span style="color: black;">无</span>创作过的、<span style="color: black;">拥有</span>文学价值的诗歌给寻找或生成出来，就很<span style="color: black;">拥有</span>挑战性。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">科研</span>方面</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/ed07fe7a4d084c87ab7cd61173547378~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=oyXSGLvACa28%2FtClTpBcvanknP4%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">这个任务<span style="color: black;">亦</span>利于<span style="color: black;">咱们</span>去<span style="color: black;">科研</span>人类的智能。<span style="color: black;">按照</span>多元智能理论，语言智能是人类智能的重要<span style="color: black;">构成</span>部分，而诗歌，尤其是中文古典诗歌，是一种非常特殊的语言形式，<span style="color: black;">拥有</span>多种优点，是<span style="color: black;">科研</span>可计算性创造力的非常好的切入点。把这两者结合在<span style="color: black;">一块</span>，利于<span style="color: black;">咱们</span><span style="color: black;">将来</span>构建真正的类人AI。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">应用方面</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/a622bb6bd63e4a77913b3646d5333df5~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=hMiPqmTvTgHdOv%2B4kK6iLIUOOzg%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">除了<span style="color: black;">科研</span>价值之外，这个研究方向<span style="color: black;">亦</span>在<span style="color: black;">许多</span><span style="color: black;">行业</span>有着丰富的应用场景和<span style="color: black;">商场</span>价值，<span style="color: black;">因此</span>在<span style="color: black;">研发</span>九歌系统之前市面<span style="color: black;">亦</span>有非常多的写诗软件。而社会各界人士，<span style="color: black;">包含</span>国内外的<span style="color: black;">有些</span>作家、诗人、大学教授，<span style="color: black;">亦</span>对<span style="color: black;">设备</span>写诗有了越来越正面的<span style="color: black;">评估</span>。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">--</span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">02</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">九歌的模型算法</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">接下来介绍<span style="color: black;">全部</span>系统里面到底用了什么样的算法，做了什么样的模型。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">1. 任务描述与<span style="color: black;">科研</span>框架</span></strong></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/27a3ae23c558430b89ca2b588da167bf~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=Bt1vR8eBHVELTI%2B2FmiKYNnzTH8%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">诗歌生成，<span style="color: black;">咱们</span><span style="color: black;">能够</span>广义地将其定义成篇章级文学性文本的<span style="color: black;">要求</span>生成任务。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p26-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/18dc550207c94f249e5342c6ee053c48~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=Iml18pybLVdhE21cZFqigsjBfhk%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">为了更好的<span style="color: black;">加强</span>生成诗歌的质量，<span style="color: black;">咱们</span>去关注中文古典诗歌的文学表现力，<span style="color: black;">由于</span>诗歌是个文学体裁，可能<span style="color: black;">区别</span>于评论和商品描述，<span style="color: black;">必须</span><span style="color: black;">更加多</span>地<span style="color: black;">重视</span>可读性和文学性。<span style="color: black;">因此</span><span style="color: black;">咱们</span>拆解了文学表现力的两个层面，一是文本质量，这是表现力的<span style="color: black;">基本</span>。二是审美特征，这是诗歌<span style="color: black;">做为</span>文学体裁区别于其他文本的最大<span style="color: black;">特殊</span>。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">文学质量方面，<span style="color: black;">咱们</span>关注连贯性、扣题性</span></strong><span style="color: black;">。</span><strong style="color: blue;"><span style="color: black;">审美特征方面，<span style="color: black;">咱们</span>关注新颖性、风格化、情感</span></strong><span style="color: black;">化。<span style="color: black;">针对</span>所有的方面，<span style="color: black;">咱们</span>都做了逐一的系统化<span style="color: black;">科研</span>，<span style="color: black;">亦</span>都发了<span style="color: black;">区别</span>的paper，并对这些技术做了工程化的实现，<span style="color: black;">最后</span>集<span style="color: black;">成为了</span><span style="color: black;">咱们</span>的九歌系统。<span style="color: black;">能够</span>看到这几个点和诗歌的广义描述是一一对应的，连贯性对应篇章性，扣题性对应<span style="color: black;">要求</span>生成，审美特征对应文学性。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">2. 针对<span style="color: black;">提高</span>文学表现力的五个层面的算法<span style="color: black;">科研</span></span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">① 连贯性</span></strong></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/553af5457f024e5f8d0b74d4b791f0cd~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=ZuFOmGeUWnPrKLcunsxZhphhNBo%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">因为</span>人类创作的写诗的句子之间都是有紧密衔接和自然过渡的，<span style="color: black;">况且</span>这些句子<span style="color: black;">做为</span>整体，其主题意境有较好的一致性。而<span style="color: black;">设备</span>生成的诗作，可能会存在前后主题不一致，<span style="color: black;">例如</span>这首春风，前两句写了一个非常和煦的初春晚冬景色，后两句变为描写边塞的怀古的非常消极的感慨，中间不存在任何过渡，连贯性并<span style="color: black;">欠好</span>。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">问题<span style="color: black;">原由</span>探究</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/52f46bdabca24336bafdfb53930aac54~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=zqC70cjSnSAYBz5Li5AMyhPKfLc%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">之<span style="color: black;">因此</span><span style="color: black;">显现</span>这个问题，<span style="color: black;">重点</span>是之前诗歌生成的<span style="color: black;">办法</span><span style="color: black;">针对</span>上文的利用方式不恰当而<span style="color: black;">导致</span>的。之前<span style="color: black;">运用</span>了单一历史（上文）向量<span style="color: black;">办法</span>，它存在三种问题如图所示。<span style="color: black;">另一</span>一种被<span style="color: black;">运用</span>的<span style="color: black;">办法</span>是拼接完整上文语句，但<span style="color: black;">咱们</span><span style="color: black;">发掘</span><span style="color: black;">倘若</span>模型的容纳能力（capacity）<span style="color: black;">不足</span>，<span style="color: black;">针对</span>长的输入序列会<span style="color: black;">显现</span><span style="color: black;">显著</span>的性能下降。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">处理</span>思路之<span style="color: black;">明显</span>性线索模型</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/ccb6d311bae54f65aa0b79502865c74b~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=KjFOe36EScF79CgoedKXnviY82U%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">为<span style="color: black;">认识</span>决这个问题，<span style="color: black;">咱们</span><span style="color: black;">首要</span>提出了一个<span style="color: black;">明显</span>性线索模型（Salient Clue Model），这个模型的灵感<span style="color: black;">源自</span>于《文心雕龙·章句》的“意脉”一词。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/814543bed04d44dc8670ac9c11debbf6~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=zJ95FnK1OXZ5IvDkLiSYMoW0P8c%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">为了实现这一点，<span style="color: black;">咱们</span>提出<span style="color: black;">明显</span>性线索模型</span></strong><span style="color: black;">，设计思路是在一首诗的生成过程中，对每一诗句丢弃其中<span style="color: black;">无</span><span style="color: black;">实质</span>语义的部分，<span style="color: black;">例如</span>虚词，并<span style="color: black;">运用</span>上文中<span style="color: black;">拥有</span><span style="color: black;">明显</span>语义的局部内容来代替完整上文，形成<span style="color: black;">明显</span>性线索以引导下文生成。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">由于</span><span style="color: black;">咱们</span><span style="color: black;">运用</span>局部上文来替代完整上文，<span style="color: black;">因此</span><span style="color: black;">能够</span>避免<span style="color: black;">太多</span>的约束。<span style="color: black;">同期</span>应用<span style="color: black;">明显</span>上文<span style="color: black;">帮忙</span><span style="color: black;">咱们</span>减少干扰，<span style="color: black;">加强</span><span style="color: black;">相关</span>。而<span style="color: black;">运用</span>动态构建<span style="color: black;">全部</span>线索骨架而非预先指定线索的方式，<span style="color: black;">能够</span><span style="color: black;">增多</span>创造性和灵活性。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/133d05f5e8dd4bcaacef287dc479bdb2~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=DfUcIJLi28uOuW6ZwDALiesH1Sk%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">这一模型的核心在于<span style="color: black;">怎样</span>计算<span style="color: black;">每一个</span>字的<span style="color: black;">明显</span>性，<span style="color: black;">咱们</span>应用了两部分信息。一部分是基于<span style="color: black;">全部</span>语料库静态计算出来的TF-IDF值，<span style="color: black;">做为</span>全局<span style="color: black;">明显</span>性。<span style="color: black;">另一</span>在生成过程中，两个句子之间的attention矩阵，<span style="color: black;">做为</span>动态局部<span style="color: black;">明显</span>性。<span style="color: black;">例如</span><span style="color: black;">咱们</span><span style="color: black;">能够</span>把这个attention矩阵按列加和，就<span style="color: black;">能够</span>得到输入端<span style="color: black;">每一个</span>字的局部<span style="color: black;">明显</span>性，之后<span style="color: black;">咱们</span>提出了<span style="color: black;">明显</span>性<span style="color: black;">选取</span>算法，进一步过滤掉里面不是<span style="color: black;">那样</span><span style="color: black;">明显</span>的词，再整合到全局<span style="color: black;">明显</span>性中做加权，最后就<span style="color: black;">能够</span>把鸿雁这个关键意象挑选出来。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/50eceaa3dab84886853eb706f340800b~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=wxv57lNtLYUaZIJ4qqyCmbO%2B92c%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span>采用自动评测和人工评测相结合的方式进行诗歌评测，自动评测采用BLEU，人工评测会找<span style="color: black;">有些</span>专家依据<span style="color: black;">区别</span>的指标按一到五分打分，<span style="color: black;">经过</span>对绝句的实验<span style="color: black;">能够</span><span style="color: black;">显示</span>诗歌的连贯性和整体质量提升最为<span style="color: black;">显著</span>，如图所示。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/92fdc8d968a742fcab07e193af64c66b~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=UHLvaiZw27IU5UhZ30Fe7Eqdllo%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">在实例中<span style="color: black;">能够</span>看出，模型能对关键意象进行<span style="color: black;">选取</span>，来引导诗歌最后一句生成高度<span style="color: black;">关联</span>的落叶这一意象。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">进一步优化之工作记忆模型</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/851b6644f110473ebaa2726a9e3392eb~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=JyjWC5JCubO2YbNFDSc4KH3dyxM%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">这个做法存在的问题是，在<span style="color: black;">每一个</span>句子中挑选多少个<span style="color: black;">明显</span>的字，这是一个人为指定的超参数。<span style="color: black;">倘若</span>挑多了会<span style="color: black;">导致</span>冗余，少了又会漏掉<span style="color: black;">有些</span>关键信息。<span style="color: black;">咱们</span>其实更倾向于要一种更加灵活动态的<span style="color: black;">办法</span>去自动决定每句中有多少个关键的<span style="color: black;">明显</span>的内容<span style="color: black;">必须</span><span style="color: black;">保存</span>，而不<span style="color: black;">必须</span>人工指定。<span style="color: black;">针对</span>这一点，<span style="color: black;">咱们</span>参考了心理语言学中关于上下文连贯性的描述，它认为只要<span style="color: black;">咱们</span>能够把内容和存在于working memory中的语义连在<span style="color: black;">一块</span>，就能够实现连贯性。working memory是人类大脑中存在的一个<span style="color: black;">拥有</span>有限容纳能力的结构，用来存储临时的信息，<span style="color: black;">能够</span>用来处理后面的决策等。<span style="color: black;">咱们</span>模拟这个working memory，提出了工作记忆模型。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/c26ccb9680fb48ebb03102cc9293f155~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=Ocb%2BP8GsJSwz%2F9hucu2CkxZjkMw%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">这个模型有<span style="color: black;">区别</span>的记忆模块，<span style="color: black;">首要</span>有一个历史记忆模块（History Memory M1），类似于前面讲的<span style="color: black;">明显</span>性线索模型，<span style="color: black;">亦</span>是从<span style="color: black;">每一个</span>生成的句子中挑<span style="color: black;">哪些</span>最<span style="color: black;">明显</span>的token写进去。但区别在于，这些模块是动态更新擦除的，在生成过程中<span style="color: black;">倘若</span>填满了，会自动<span style="color: black;">选取</span>比较老的，比较不<span style="color: black;">关联</span>的给覆盖掉，以此<span style="color: black;">咱们</span>能<span style="color: black;">守护</span>相互独立、有限、多个记忆槽，既能<span style="color: black;">供给</span>足够的容纳能力去<span style="color: black;">守护</span>部分远距离信息，又不会无限的膨胀。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/7f7658fae02b41758da1606806342851~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=WdnSUgr7CVPmPe1YeFnjlCHUPN4%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">还有一个局部记忆模块（Local Memory M2），用来存储上一句生成的诗句，充当一种完整近距离上文信息，以此来促进对仗句等强语义<span style="color: black;">相关</span>内容的生成。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/f1ad49e9184a484ca4eb3c3a41a03845~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=S44awAR7ZkvNP%2BaR708AV4SCIQE%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">在做记忆模块读取的时候，所有<span style="color: black;">区别</span>模块会统一读取，模型就<span style="color: black;">能够</span>自动决定要关注近距离的还是远距离的<span style="color: black;">区别</span>的信息，从而<span style="color: black;">守护</span>了整首诗<span style="color: black;">留意</span>境上的<span style="color: black;">相关</span>性。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">② 扣题性</span></strong></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/e4f40e08920f4d3eb88597ac9bdfc5e3~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=LgHe7ocUcS3q9zr75ahz%2B5cYyV4%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">另一</span>一个文本质量的层面是扣题性，<span style="color: black;">由于</span>诗歌生成是一种<span style="color: black;">要求</span>生成任务，<span style="color: black;">咱们</span><span style="color: black;">期盼</span>输入的内容都<span style="color: black;">能够</span>在生成的诗歌中得到完整的体现来<span style="color: black;">提高</span>用户体验。传统诗歌生成会存在遗漏部分输入信息的现象，这<span style="color: black;">重点</span>是<span style="color: black;">因为</span><span style="color: black;">针对</span>关键词的应用方式不恰当而<span style="color: black;">导致</span>的。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">传统有两种<span style="color: black;">办法</span>来应用关键词，一种是多个关键词压缩入单一主题向量，用以引导生成，（向量中）关键词混杂会<span style="color: black;">引起</span>关键词<span style="color: black;">包括</span>率下降。另一种是关键词逐句<span style="color: black;">插进</span>，这种方式的<span style="color: black;">坏处</span>端在于当输入关键词数<span style="color: black;">少于</span>句子数时，<span style="color: black;">必须</span>应用关键词扩展机制，<span style="color: black;">况且</span>生成的诗歌过分依赖于用户输入的关键词词序<span style="color: black;">引起</span>了诗歌生成<span style="color: black;">不足</span>灵活。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">处理</span>思路之主题记忆模块与主题<span style="color: black;">跟踪</span>机制</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/15125411642f4aec8bc52bacda45df6e~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=Dh677Mzv9Lr8c9NFy1Kd10PknJs%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span>做的工作，是在工作记忆模型中加入了一个新的模块，主题记忆模块（Topic Memory M3），它会显式且独立地存储每一个主题词。<span style="color: black;">由于</span>这些关键词是显式存储读取的，关键词不会混杂，关键词的<span style="color: black;">包括</span>率就会<span style="color: black;">提升</span>。<span style="color: black;">同期</span>独立存储读取，<span style="color: black;">寓意</span>不存在相互之间词序上的依赖，这让生成诗歌的主题表达的形式和<span style="color: black;">次序</span>都更加灵活。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">同期</span><span style="color: black;">咱们</span>还设计了一个主题<span style="color: black;">跟踪</span>机制（Topic Trace），在生成过程中用更加显式的方式来记录<span style="color: black;">每一个</span>主题表达与否，以此来<span style="color: black;">加强</span>其覆盖率。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/e962102e11c94802a7d148046c58e4ba~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=f9BF1v3bQ0lJoUTxvmj0BPyUVhk%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">经过</span><span style="color: black;">针对</span>绝句的测试，平均<span style="color: black;">来讲</span><span style="color: black;">咱们</span>能做到输入关键词83%都能在诗歌中生成出来，远超之前的几个baseline model。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">用户输入为语句/段落时<span style="color: black;">显现</span>的新问题</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/54341ab4acb749dd998c4e34ba3f9dd3~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=ouc%2Bs9jMNEg%2Fuh989WTjJopH76U%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">除了关键词之外，用户还倾向于输入一个完整的语句<span style="color: black;">或</span>段落来<span style="color: black;">暗示</span>主题。<span style="color: black;">怎样</span>去处理呢？在工程化上的pipeline<span style="color: black;">便是</span>先做中文分词，之后做关键词抽取，再把抽取得到的多个关键词输入模型生成。但这个pipeline中分词和抽取都有误差，<span style="color: black;">况且</span>肯定会有信息损失，这会<span style="color: black;">引起</span><span style="color: black;">咱们</span><span style="color: black;">没法</span>生成<span style="color: black;">目的</span>诗句。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span><span style="color: black;">发掘</span>现代文和古诗文的词表有78%的重合，两者<span style="color: black;">能够</span>看成同一种语言上面的两种<span style="color: black;">区别</span>的风格。<span style="color: black;">咱们</span>提出的<span style="color: black;">处理</span><span style="color: black;">方法</span>是：文本风格转换——把用户的输入直接转换为古典诗句，以此最大程度的<span style="color: black;">保存</span>用户<span style="color: black;">供给</span>的主题信息。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">处理</span><span style="color: black;">方法</span>之实例支撑的风格转换模型</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/b10b629400d74764815ba54b37748d57~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=cqA0a4QJnmjWAkUUD9fTT0F1nnU%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">文本风格转换之前有两种<span style="color: black;">区别</span>的范式，这两者在内容<span style="color: black;">保留</span>度和风格转换准确率上各有千秋，<span style="color: black;">咱们</span>的做法是结合两种<span style="color: black;">办法</span>来取长补短。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/545bea66e7624468bf081cc40e3edfb1~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=0WZAMhimxQzTVirTb1c57%2B8jRa4%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">首要</span><span style="color: black;">咱们</span>提出了基于attention的Seq2Seq结构以此来完整地<span style="color: black;">保存</span>用户输入的词级别的源端信息，<span style="color: black;">同期</span><span style="color: black;">咱们</span><span style="color: black;">运用</span>隐空间风格<span style="color: black;">暗示</span>来构造更<span style="color: black;">拥有</span>区分度和表达能力的风格信号，最后将两者结合在<span style="color: black;">一块</span>，从而实现了转化之后用户输入的语义的内容<span style="color: black;">保存</span>度以及转化后诗歌风格准确度的更好的平衡。<span style="color: black;">咱们</span>推导了新的数学形式，<span style="color: black;">首要</span><span style="color: black;">运用</span>一组风格实例，例如<span style="color: black;">运用</span>100个诗句这个小的集合来<span style="color: black;">表率</span>古诗这种风格，<span style="color: black;">亦</span><span style="color: black;">便是</span>该特定风格的经验分布。基于此推导出了新的转化形式，这和<span style="color: black;">咱们</span>的模型结构是一一对应的。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/973d2d8e072b4416b1a49c34f647eeb6~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=4N2xQUoGdt5Ut8MiXE3aNG%2BcOAs%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span>的核心是风格编码器，而问题在于<span style="color: black;">怎样</span>从这组风格实例中抽取一个足够灵活和有表征能力的风格<span style="color: black;">暗示</span>信号。这个过程是在隐空间进行，<span style="color: black;">运用</span>生成式流模型，<span style="color: black;">咱们</span>会构造一个更加<span style="color: black;">繁杂</span>的风格隐空间<span style="color: black;">而后</span>从里面去做sample。<span style="color: black;">另一</span><span style="color: black;">针对</span>这种古诗和现代汉语的转化，<span style="color: black;">咱们</span>是有一部少量的翻译标注数据的，<span style="color: black;">因此</span>为了有效利用这些数据，<span style="color: black;">咱们</span>进一步推出了一个半监督的训练损失如图所示。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span>在优化该损失的时候是在<span style="color: black;">同期</span>做三件事情，<span style="color: black;">首要</span>是在最大化由<span style="color: black;">咱们</span>的现代汉语文本和风格实例所生成<span style="color: black;">目的</span>风格诗句的概率的下界。<span style="color: black;">同期</span>在最小化这个概率的负值的上界，向两边逼近它。最后在对齐<span style="color: black;">有没有</span>标注数据时的古诗的分布，以此在<span style="color: black;">最后</span>生成时学到更加符合真实的诗歌空间的风格<span style="color: black;">暗示</span>。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">③ 新颖性</span></strong></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/e6916158984d42c091ed6869235b1ad3~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=uSeOJxLBX%2BHqm0WORVQjINXOlmA%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">除了文本质量之外，诗歌<span style="color: black;">做为</span>一种文学性文本，它最大的特点<span style="color: black;">便是</span>审美特征，<span style="color: black;">首要</span>是新颖性。<span style="color: black;">由于</span>用户<span style="color: black;">期盼</span>读到新颖有趣的诗歌，而不是重复乏味无聊<span style="color: black;">况且</span>被人写过的诗作。<span style="color: black;">针对</span>新颖性<span style="color: black;">咱们</span>最低的<span style="color: black;">需求</span>是，<span style="color: black;">针对</span>用户<span style="color: black;">区别</span>的主题词输入，模型能够生成有差异性和新颖的诗歌。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">因为</span><span style="color: black;">咱们</span><span style="color: black;">通常</span><span style="color: black;">运用</span>MLE<span style="color: black;">极重</span>似然估计来做模型的优化，这<span style="color: black;">引起</span>了上面所述<span style="color: black;">需求</span>很难实现。<span style="color: black;">详细</span><span style="color: black;">来讲</span><span style="color: black;">便是</span>一个Token级别交叉熵损失。这种损失是倾向于记忆并生成高频模式，<span style="color: black;">例如</span>高频n-grams、停用词。<span style="color: black;">同期</span>这种损失函数的<span style="color: black;">评估</span>指标在<span style="color: black;">评估</span>粒度和<span style="color: black;">评估</span>指标上都与人类的<span style="color: black;">评估</span>不匹配。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">处理</span>思路之互强化学习</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/6cfdadc9ac154bc8b8fe636ce0e98ca9~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=8fx5V%2B5un5nOKwGLMeUlGxwcpsI%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">为<span style="color: black;">认识</span>决这个问题<span style="color: black;">咱们</span>提出<span style="color: black;">运用</span>强化学习，<span style="color: black;">咱们</span>对人类<span style="color: black;">评估</span>诗歌的每一个指标都做了量化的近似和建模，用这些来<span style="color: black;">做为</span>一个评分性的rewarder，用强化学习去激励模型在训练过程中去生成在这些指标上能得到更高得分的诗作。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/895a814647ab49af9d582c0966b80007~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=jD3D2lmCFXlfHNPfC26y6IUuS5g%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">进一步<span style="color: black;">咱们</span>提出了互强化学习，<span style="color: black;">由于</span>写作学习<span style="color: black;">针对</span>人<span style="color: black;">来讲</span>是一种群体性的任务，交流非常重要，有必要<span style="color: black;">准许</span>训练过程中生成器之间有<span style="color: black;">必定</span>的交流和借鉴。<span style="color: black;">因此呢</span><span style="color: black;">咱们</span>在训练过程中，<span style="color: black;">同期</span>训练两个不同的生成器来模拟学生，打分器来模拟老师，生成器不仅从老师那获取梯度反馈信号，<span style="color: black;">同期</span>相互之间<span style="color: black;">亦</span>会有<span style="color: black;">必定</span>的信息交互。为了实现这种交互<span style="color: black;">咱们</span>提出了一种算法，<span style="color: black;">能够</span>在<span style="color: black;">全部</span>强化学习的策略空间搜索时沿着<span style="color: black;">区别</span>的两条路径来搜索，既能加快搜索速度，又能避免某个生成器陷入局部最优点。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">④ 风格化</span></strong></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/f66ba4a4313f48efb47e9aa75bc787e8~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=7%2B%2Fsosl66cadS2OpTQXksi07W3o%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">另一</span>一个特点，审美特征是风格化。<span style="color: black;">咱们</span><span style="color: black;">晓得</span>人类在同一个主题下是有能力创作出完全<span style="color: black;">区别</span>风格的诗歌的，<span style="color: black;">因此</span><span style="color: black;">咱们</span>想让模型<span style="color: black;">亦</span>实现风格的<span style="color: black;">掌控</span>。<span style="color: black;">咱们</span>要做的是把<span style="color: black;">无</span>风格区分度的<span style="color: black;">全部</span>诗歌空间p（x）做解耦合到<span style="color: black;">区别</span>的风格依赖的风格上子空间上，<span style="color: black;">这般</span>就<span style="color: black;">能够</span><span style="color: black;">选取</span>对应风格的空间，从中生成<span style="color: black;">咱们</span>想要风格的诗歌。但<span style="color: black;">咱们</span><span style="color: black;">仅有</span>很少的标注数据，<span style="color: black;">因此</span>想要无监督地去实现这一目标。<span style="color: black;">咱们</span><span style="color: black;">无</span>办法去构建<span style="color: black;">要求</span>概率分布，<span style="color: black;">亦</span><span style="color: black;">便是</span><span style="color: black;">无</span>办法构建生成诗歌与输入的风格label之间的<span style="color: black;">相关</span>关系。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">处理</span>思路之无监督学习与利用正则项添加<span style="color: black;">相关</span>性</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/7b6918b37e9d49fb9c8f157a5e713aa0~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=URg7dCQJosi5wipN0GE6NIM%2FvrM%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">处理</span><span style="color: black;">方法</span>是用一个正则项去强加这个关系。<span style="color: black;">详细</span><span style="color: black;">来讲</span>，<span style="color: black;">便是</span>最大化风格分布和诗歌分布之间的互信息，强行把这种依赖给加上。互信息是衡量两个变量之间的依赖程度的。加上之后，改变风格标签y，生成的诗歌x就会跟着改变，以此实现了风格的<span style="color: black;">掌控</span>。</span></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/05d2fcf2789c472ebc26c395bf21673f~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=vvt0%2Fb8vFV9j8reOKvJWdQXvuVI%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span>得出了<span style="color: black;">这般</span>的损失函数，由两部分<span style="color: black;">构成</span>，一部分是风格无关的似然项，它能够<span style="color: black;">保证</span><span style="color: black;">咱们</span>生成的诗歌是比较通畅的，和上文<span style="color: black;">相关</span>性比较好。<span style="color: black;">另一</span>一个是风格的正则项，用来给输入的风格标签和输出的诗歌的文本空间强加一个<span style="color: black;">相关</span>性，以此来实现<span style="color: black;">掌控</span>。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">⑤ 情感化</span></strong></span></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/e3b8b7c84cab4653ad23998a3d7886b7~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=j5iHSZMLs4Ie7QzP4V5tdh%2BrlnY%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">最后一个审美特征是情感化，抒情是人类诗歌创作最<span style="color: black;">重点</span>的目的之一。<span style="color: black;">咱们</span>想让生成模型<span style="color: black;">亦</span>做到情感的<span style="color: black;">掌控</span>。人类写诗的情感表达有两大特点，<span style="color: black;">首要</span>是<span style="color: black;">针对</span>同一个主题人们能写出两种<span style="color: black;">区别</span>的情感。其次一首诗歌内部每一句的情感并不完全相同，<span style="color: black;">拥有</span><span style="color: black;">必定</span>的变化转折规律。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;"><span style="color: black;">处理</span>思路之提出新拆解式</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/1f5ae6a9a0cb447fb6ba27cc4c9139fa~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=zfUfsJ%2B4kK9Qug7J8WkVJsPgWDI%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">模型思路：针对文学文本<span style="color: black;">咱们</span>提出了一个全新的拆解式，来描述<span style="color: black;">咱们</span>的生成模型是怎么生成一首诗歌的。<span style="color: black;">首要</span><span style="color: black;">咱们</span>的user给指定一个主题词w，<span style="color: black;">亦</span><span style="color: black;">能够</span>指定一个情感标签y，若其不指定，<span style="color: black;">咱们</span><span style="color: black;">亦</span><span style="color: black;">能够</span>自动的去预测一个y。y和w<span style="color: black;">一起</span>构<span style="color: black;">成为了</span><span style="color: black;">咱们</span>的一个隐空间，这三部分<span style="color: black;">一起</span>用于生成<span style="color: black;">咱们</span>的诗歌。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">针对全诗情感采用半监督循环训练</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/81c3dd47ce594b85b82285a9a7e248ce~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=E%2FtIKGa1Cgh3r%2FXt2409CPHxTuU%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">当<span style="color: black;">咱们</span>要<span style="color: black;">掌控</span>整首诗的整体情感时，<span style="color: black;">倘若</span>有标注数据，比较简单。<span style="color: black;">咱们</span>会按标准的流程去推导出它的ELBO下界来直接优化。区别在于<span style="color: black;">咱们</span>天然的自带了一个<span style="color: black;">归类</span>器，这个<span style="color: black;">归类</span>器会依据<span style="color: black;">咱们</span>的主题词自动地预测一个最合适的情感标签。<span style="color: black;">无</span>label的时候，<span style="color: black;">咱们</span>把情感y看成<span style="color: black;">另一</span>一个隐变量，又<span style="color: black;">能够</span>推出它的<span style="color: black;">另一</span>一项，<span style="color: black;">亦</span>进行优化。这<span style="color: black;">其中</span>的第二个<span style="color: black;">归类</span>器<span style="color: black;">能够</span>用来为<span style="color: black;">每一个</span>无标注的诗歌预测一个适合的情感。<span style="color: black;">咱们</span>把两者结合在<span style="color: black;">一块</span>。在<span style="color: black;">全部</span>训练过程中，用<span style="color: black;">哪些</span>有标注的数据去训练模型和<span style="color: black;">归类</span>器，<span style="color: black;">归类</span>器反过来为<span style="color: black;">哪些</span>无标注的数据预测一个合适的标签，<span style="color: black;">持续</span>循环交替迭代，以此来实现半监督的训练。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">针对每句情感采用交叉时间序列进行训练</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/0f7e11dd8af9491a914fdf19026030af~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=MHKAHtYdaBHnSD0URgvQFDwLGoY%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">倘若</span><span style="color: black;">咱们</span>想要<span style="color: black;">掌控</span>每一句的情感，<span style="color: black;">咱们</span><span style="color: black;">思虑</span><span style="color: black;">每一个</span>诗句xi和这个诗句的句子级别的情感yi，无label的<span style="color: black;">状况</span>下，推出的式子中有一个期望，将其用蒙特卡罗采样，会产生一个时间序列的拆解，有一个序列的采样。在这个过程中<span style="color: black;">同期</span>学两个<span style="color: black;">区别</span>的序列：一个是<span style="color: black;">每一个</span>句子的情感<span style="color: black;">形成</span>，<span style="color: black;">另一</span>一个是<span style="color: black;">每一个</span>诗句si的内容的<span style="color: black;">次序</span>，<span style="color: black;">咱们</span>是在<span style="color: black;">同期</span>对两者进行建模和学习。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">另一</span>，虽然九歌算法诞生的时间比较早，但以上算法均<span style="color: black;">能够</span>移植到最新的模块上。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">--</span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">03</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">九歌系统介绍</span></strong></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/5c5e6d1e2d694b1993f9a360c5784e44~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=eifrKreJ64L43gcPmknChQVtxcc%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">以上所有的工作都做了工程化的实现，并整合到了中文古典诗歌在线生成系统——九歌中，在线系统网址（</span><span style="color: black;">jiuge.thunlp.org/</span><span style="color: black;">），<span style="color: black;">大众</span><span style="color: black;">能够</span>到网站上体验九歌系统的功能。九歌系统累计为用户创作诗歌超过2500万首，用户遍布<span style="color: black;">全世界</span>，<span style="color: black;">亦</span>为万里之外漂泊的祖国游子<span style="color: black;">供给</span>了一点小小的慰藉，这<span style="color: black;">亦</span>是<span style="color: black;">咱们</span>做文化与AI相结合的初衷之一。<span style="color: black;">全部</span>项目<span style="color: black;">得到</span>了一系列奖项，系统及其创作的诗作在《机智过人》<span style="color: black;">第1</span>季和《朗读者》节目、人工智能教育大会等场合进行过展示，成果被多家<span style="color: black;">媒介</span>广泛<span style="color: black;">报告</span>，有<span style="color: black;">必定</span>的社会影响，并与学堂在线、腾讯相册管家进行合作。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">--</span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">04</span></strong></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">自动作诗与知识图谱</span></strong></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/8f43020c75354cecb72649f3535216ba~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=fySmhi1yrH9CVMJickfjC6VlZ58%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">咱们</span><span style="color: black;">亦</span>有用到一部分知识图谱的知识，<span style="color: black;">咱们</span>构建了概率<span style="color: black;">相关</span>式的知识图谱——文脉，爬取维基百科中所有entity的链接，计算出一个带边权的链接的网络，如图所示，并将这个知识图谱map到了古诗文上，最后得到在所有古诗文中<span style="color: black;">显现</span>的<span style="color: black;">这般</span>一个带边权的图谱,所有权重会用词云的方式<span style="color: black;">表现</span>出来。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在线演示</p>:https://williamlwclwc.github.io/KG-Demo/
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">开源下载</p>:https://github.com/THUNLP-AIPoet/ParCKG
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">除了常规的关键词扩展和转换以<span style="color: black;">帮忙</span><span style="color: black;">设备</span>理解词语外，<span style="color: black;">咱们</span>还对其有一系列的<span style="color: black;">将来</span>展望：</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">常识驱动的用户输入理解</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/5060bf494e85491c9c9e00b2587c9cae~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=erCIZXpWJn2PnQWV%2FIN4NqGnSbo%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">人<span style="color: black;">能够</span>用古典的语言去描述现代的“飞机”一词，<span style="color: black;">由于</span>人有常识。而<span style="color: black;">日前</span>的绝大<span style="color: black;">都数</span>模型<span style="color: black;">没法</span>做到这一点，图谱的应用会<span style="color: black;">帮忙</span>改善这一<span style="color: black;">状况</span>。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">引入语言学与文学知识的诗歌生成</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/e638830c13794e31bd47d1d1b887f78a~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=sPTJbHtC7DiX1MMugNvezT4qQNg%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">知识<span style="color: black;">能够</span>用来<span style="color: black;">帮忙</span>实现将典故知识输入模型来促进诗歌生成，<span style="color: black;">或</span><span style="color: black;">运用</span><span style="color: black;">各样</span>修辞来<span style="color: black;">促进</span>模型更好地生成更有寓意的诗歌。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">结合时空及历史知识的诗歌生成</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/5c9a50f876bf462fbdebfb6fb91e81fd~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=4Qs8%2Fx5nAU9bHvO%2BJeX2uYrhgBQ%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">古人作诗常常是登高望景，所见即所得，<span style="color: black;">因此</span>诗歌中常含有地理空间的<span style="color: black;">原因</span>以及历史知识的融入。</span></span></span></p><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">AI与人类的关系之我见</span></strong></span></span>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/179207b4bdbf4b1195c7ab91fac8a73c~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=uESXK2ImVTWpP5orJ1RWD%2BZyfsg%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">此刻</span><span style="color: black;">咱们</span>用人类几百年来沉淀下来的作品来<span style="color: black;">指点</span>AI学习，但随着技术的发展，AI会创作出更好的诗歌，以此激励人类创作者创作出新的诗歌，反过来进一步<span style="color: black;">提高</span>AI，形成良性循环。创作上，<span style="color: black;">设备</span>与人不是非此即彼、相互取代，而是相互促进<span style="color: black;">一起</span>学习和进步的，这<span style="color: black;">亦</span>是九歌系统<span style="color: black;">开发</span>的初衷之一。<span style="color: black;">期盼</span><span style="color: black;">将来</span><span style="color: black;">咱们</span>能够<span style="color: black;">一块</span>在这个方向上做<span style="color: black;">更加多</span>更有意思的探索和尝试。谢谢各位！</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">今天的分享就到<span style="color: black;">这儿</span>，谢谢<span style="color: black;">大众</span>。</span></strong></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">阅读<span style="color: black;">更加多</span>技术干货<span style="color: black;">文案</span>、下载讲师PPT，请关注</span></span><strong style="color: blue;"><span style="color: black;"><span style="color: black;">微X</span>公众号“DataFunTalk”。</span></strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">分享嘉宾：矣晓沅博士 微软亚洲<span style="color: black;">科研</span>院 <span style="color: black;">科研</span>员</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">编辑整理：赵洪业 合肥工业大学</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">出品平台：DataFunTalk</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">分享嘉宾：</span></strong></span></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/334d1732e43841d9a8844564d4a24378~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723947332&x-signature=SmDyZZ7yz46TQAt%2BFgboKVhESIs%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">关于<span style="color: black;">咱们</span>：</span></strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">DataFun：</span></strong><span style="color: black;">专注于大数据、人工智能技术应用的分享与交流。发起于2017年，在北京、上海、深圳、杭州等城市举办超过100+线下和100+线上沙龙、论坛及峰会，已邀请超过2000位专家和学者参与分享。其公众号 DataFunTalk 累计生产原创<span style="color: black;">文案</span>700+，百万+阅读，14万+<span style="color: black;">精细</span>粉丝。</span></span></span></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">欢迎转载分享评论，转载请私信。</span></span></p>
我深受你的启发，你的话语是我前进的动力。 对于这个问题，我有不同的看法... 外链发布论坛学习网络优化SEO。 谢谢、感谢、感恩、辛苦了、有你真好等。 这夸赞甜到心里，让我感觉温暖无比。
页:
[1]