6257rv7 发表于 2024-8-31 05:48:31

【Python深度学习系列】网格搜索神经网络超参数：优化器


    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">这是我的</span></span><span style="color: black;"><strong style="color: blue;"><span style="color: black;">第286篇</span></strong></span><span style="color: black;"><span style="color: black;">原创<span style="color: black;">文案</span>。</span></span></p>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;"><span style="color: black;">1、</span>引言</h1>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-axegupay5k/a553b28e02c34790900df8cc4e92556e~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725628880&amp;x-signature=2NBCvVBKn%2FwL7d5%2FxoLHnwb2ZB4%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">在深度学习中，超参数<span style="color: black;">指的是</span>在训练模型时需要手动设置的参数，它们<span style="color: black;">一般</span><span style="color: black;">不可</span><span style="color: black;">经过</span>训练数据自动学习得到。超参数的<span style="color: black;">选取</span><span style="color: black;">针对</span>模型的性能至关重要，<span style="color: black;">因此呢</span>在进行深度学习实验时，超参数调优<span style="color: black;">一般</span>是一个重要的<span style="color: black;">过程</span>。<span style="color: black;">平常</span>的超参数<span style="color: black;">包含</span>：</span></span></p><span style="color: black;"><span style="color: black;">model.add()</span></span><span style="color: black;"><span style="color: black;">neurons（隐含层神经元数量）</span></span><span style="color: black;"><span style="color: black;">init_mode（初始权重<span style="color: black;">办法</span>）</span></span><span style="color: black;"><span style="color: black;">activation（激活函数）</span></span><span style="color: black;"><span style="color: black;">dropout（丢弃率）</span></span><span style="color: black;"><span style="color: black;">model.compile()</span></span><span style="color: black;"><span style="color: black;">loss（损失函数）</span></span><span style="color: black;"><span style="color: black;">optimizer（优化器）</span></span><span style="color: black;">learning rate（学习率）</span><span style="color: black;">momentum（动量）</span><span style="color: black;">weight decay（权重衰减系数）</span><span style="color: black;"><span style="color: black;">model.fit()</span></span><span style="color: black;"><span style="color: black;">batch size（批量<span style="color: black;">体积</span>）</span></span><span style="color: black;"><span style="color: black;">epochs（迭代次数）</span></span>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">通常</span><span style="color: black;">来讲</span>，<span style="color: black;">能够</span><span style="color: black;">经过</span>手动调优、网格搜索（Grid Search）、随机搜索（Random Search）、自动调参算法方式进行超参数调优。</span><span style="color: black;">损失函数是用于衡量模型所作出的预测离真实值（Ground Truth）之间的偏离程度。<span style="color: black;">一般</span>，<span style="color: black;">咱们</span>都会用一种</span><strong style="color: blue;"><span style="color: black;">优化算法（优化器）</span></strong><span style="color: black;">最小化<span style="color: black;">目的</span>函数，<span style="color: black;">例如</span>常用的算法便是“梯度下降法”（Gradient Descent）。</span><span style="color: black;">本文采用网格搜索<span style="color: black;">选取</span>优化器。</span></span></p>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;"><span style="color: black;">2、</span>实现过程</h1>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;">2.1 准备数据</h1>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">dataset：</span></span></span></p>dataset = pd.read_csv(<span style="color: black;">"data.csv"</span>, header=None)
    dataset = pd.DataFrame(dataset)
    print(dataset)<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-6w9my0ksvp/85a67d8c3a3c492cb183e1269b62f87f~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725628880&amp;x-signature=QZBCi0tUkao4IkgEPyYgvIN9dKI%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;">2.2 数据划分</h1><span style="color: black;"># 切分数据为输入 X 和输出 Y</span>
    X = dataset.iloc[:,0:8]
    Y = dataset.iloc[:,8]
    <span style="color: black;"># 为了复现，设置随机种子</span>seed = 7
    np.random.seed(seed)
    random.set_seed(seed)<h1 style="color: black; text-align: left; margin-bottom: 10px;">2.3 创建模型</h1>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">需要定义个网格的架构函数create_model</span></span></span></p><span style="color: black;">def <span style="color: black;">create_model</span>():
      # 创建模型
      model</span> = Sequential()
    model.<span style="color: black;">add</span>(Dense(<span style="color: black;">50</span>, input_shape=(<span style="color: black;">8</span>, ), kernel_initializer=<span style="color: black;">uniform</span>, activation=<span style="color: black;">relu</span>))
    model.<span style="color: black;">add</span>(Dropout(<span style="color: black;">0.05</span>))
    model.<span style="color: black;">add</span>(Dense(<span style="color: black;">1</span>, kernel_initializer=<span style="color: black;">uniform</span>, activation=<span style="color: black;">sigmoid</span>))

    <span style="color: black;"># 编译模型</span>
    model.compile(loss=<span style="color: black;">binary_crossentropy</span>, optimizer=<span style="color: black;">adam</span>, metrics=[<span style="color: black;">accuracy</span>])
    <span style="color: black;">return</span>model
    model = KerasClassifier(model=create_model, epochs=<span style="color: black;">100</span>, batch_size=<span style="color: black;">80</span>, verbose=<span style="color: black;">0</span>)<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">这儿</span><span style="color: black;">运用</span>了scikeras库的KerasClassifier类来定义一个<span style="color: black;">归类</span>器，<span style="color: black;">这儿</span><span style="color: black;">因为</span>KerasClassifier有定义优化器的参数</span></span></span><span style="color: black;"><span style="color: black;"><span style="color: black;">optimizer</span></span></span><span style="color: black;"><span style="color: black;">，不需要自定义一个<span style="color: black;">暗示</span>丢优化器的参数。</span></span></p>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;">2.4 定义网格搜索参数</h1><span style="color: black;">param_grid</span> = {<span style="color: black;">optimizer</span>: [<span style="color: black;">SGD</span>, <span style="color: black;">RMSprop</span>, <span style="color: black;">Adagrad</span>, <span style="color: black;">Adadelta</span>, <span style="color: black;">Adam</span>, <span style="color: black;">Adamax</span>, <span style="color: black;">Nadam</span>]}<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">param_grid是一个字典，key是超参数名<span style="color: black;">叫作</span>，<span style="color: black;">这儿</span>的名<span style="color: black;">叫作</span>必须要在KerasClassifier这个对象里面存在<span style="color: black;">况且</span>参数名要一致。value是key可取的值，<span style="color: black;">亦</span><span style="color: black;">便是</span>要尝试的<span style="color: black;">方法</span>。</span></span></span></p>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;">2.5 进行参数搜索</h1><span style="color: black;">from</span> <span style="color: black;">sklearn.model_selection import GridSearchCV</span>
    <span style="color: black;">grid</span> = <span style="color: black;">GridSearchCV(estimator=model, param_grid=param_grid)</span>
    <span style="color: black;">grid_result</span> = <span style="color: black;">grid.fit(X, Y)</span>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;"><span style="color: black;">运用</span>sklearn里面的GridSearchCV类进行参数搜索，传入模型和网格参数。</span></span></span></p>
    <h1 style="color: black; text-align: left; margin-bottom: 10px;">2.6 总结搜索结果</h1><span style="color: black;">print</span>(<span style="color: black;">"Best: %f using %s"</span> % (grid_result.best_score<span style="color: black;">_</span>, grid_result.best_params<span style="color: black;">_</span>))
    means = grid_result.cv_results<span style="color: black;">_</span>[<span style="color: black;">mean_test_score</span>]
    stds = grid_result.cv_results<span style="color: black;">_</span>[<span style="color: black;">std_test_score</span>]
    params = grid_result.cv_results<span style="color: black;">_</span>[<span style="color: black;">params</span>]
    <span style="color: black;">for</span>mean, stdev, param in zip(means, stds, params):<span style="color: black;">print</span>(<span style="color: black;">"%f (%f) with: %r"</span> % (mean, stdev, param))<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">结果：</span></span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-6w9my0ksvp/4e171d1f7a4d454c819efa21974c5cef~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725628880&amp;x-signature=4Pga9mF2haomZnfzr48Nn69CFw0%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">经过网格搜索，优化器的最优<span style="color: black;">选取</span>是Nadm。</span></span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">作者简介： 读研<span style="color: black;">时期</span><span style="color: black;">发布</span>6篇SCI数据算法<span style="color: black;">关联</span>论文，<span style="color: black;">日前</span>在某<span style="color: black;">科研</span>院从事数据算法<span style="color: black;">关联</span><span style="color: black;">科研</span>工作，结合<span style="color: black;">自己</span><span style="color: black;">研究</span>实践经历<span style="color: black;">连续</span>分享关于Python、数据分析、特征工程、<span style="color: black;">设备</span>学习、深度学习、人工智能系列<span style="color: black;">基本</span>知识与案例。关注gzh：数据杂坛，获取数据和源码学习<span style="color: black;">更加多</span>内容。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">原文链接：</strong><a style="color: black;">【Python深度学习系列】网格搜索神经网络超参数：优化器（案例+源码）</a></p>




b1gc8v 发表于 2024-8-31 09:25:26

楼主的文章非常有意义，提升了我的知识水平。

4zhvml8 发表于 2024-10-25 16:54:25

我深受你的启发，你的话语是我前进的动力。
页: [1]
查看完整版本: 【Python深度学习系列】网格搜索神经网络超参数：优化器