论坛 › 香港 › 什么是“爬虫”？

j8typz 发表于 2024-8-25 17:32:38

什么是“爬虫”？

    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">大众</span>都<span style="color: black;">晓得</span>，互联网<span style="color: black;">是由于</span>海浪<span style="color: black;">形成</span>的，<span style="color: black;">由于</span>人人都<span style="color: black;">能够</span>在里面冲浪。</span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/e7914a689fcb4033b7fa62691a1424f7~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725101967&amp;x-signature=6fpM%2BUCK8BUzObZEHiLjJmWlYCk%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">但<span style="color: black;">非常多</span>人不<span style="color: black;">晓得</span>的是，相比于浪花，互联网上最多的东西，其实是爬虫。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">言下之意<span style="color: black;">便是</span>你的网线生虫了。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">爬虫其实是一门计算机中的技术，它被广泛应用于搜索引擎。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">平常</span>的搜索引擎有百度、Google、Bing等。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">搜索引擎的工作原理大致分为爬取信息、存储、<span style="color: black;">创立</span>索引、排序、检索等环节，其中<span style="color: black;">第1</span><span style="color: black;">周期</span><span style="color: black;">便是</span><span style="color: black;">运用</span>专用程序收集网页数据，这个程序<span style="color: black;">一般</span><span style="color: black;">叫作</span>为蜘蛛（Spider）或爬虫（Crawler）。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">搜索引擎从已知的数据库出发，<span style="color: black;">拜访</span>这些网页并抓取文件。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">搜索引擎<span style="color: black;">经过</span>这些爬虫从一个网站爬到另一个网站，跟踪网页中的链接，<span style="color: black;">拜访</span><span style="color: black;">更加多</span>的网页，这个过程<span style="color: black;">叫作</span>为爬行，这些新的网址会被存入数据库等待搜索。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">简而言之，爬虫<span style="color: black;">便是</span><span style="color: black;">经过</span>不间断地<span style="color: black;">拜访</span>互联网，<span style="color: black;">而后</span>从中获取你指定的信息并返回给你。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">而<span style="color: black;">咱们</span>的互联网上，随时都<span style="color: black;">有没有</span>数的爬虫在爬取数据，并返回给<span style="color: black;">运用</span>者。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">除了搜索引擎，爬虫还<span style="color: black;">能够</span>完成<span style="color: black;">非常多</span>别的事情。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">2018年春运<span style="color: black;">时期</span>，12306网站<span style="color: black;">拜访</span>量的日峰值一度达到了1500亿，平均<span style="color: black;">每一个</span>中国人一天<span style="color: black;">拜访</span>了近100次12306网站。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">可是你转念一想，就算是抢票，<span style="color: black;">好似</span><span style="color: black;">亦</span>没见过谁一天上一百多次12306啊！</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">其实12306超高<span style="color: black;">拜访</span>量的幕后黑手，<span style="color: black;">便是</span>传说中的</span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/96c2ae89708e41bbada3e3583df49098~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725101967&amp;x-signature=SqSDjEPhX%2Fq5r1E2rWUw1Zaib6Q%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">当你在抢票APP上点击抢票软件过后，APP上就会<span style="color: black;">表示</span>抢票次数。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">事实上，APP每抢一次票就等于是利用爬虫<span style="color: black;">拜访</span>了一次12306。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">倘若</span>有余票的话，它就直接帮你拍下。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">倘若</span>没有余票，那就过一会儿再来。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">此时的爬虫就像一个死皮赖脸的油腻男，傍着12306这个如花似玉的小姑娘死活不撒手。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">爬虫：美女，有票吗~~~？</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">12306：<span style="color: black;">无</span>，滚！</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">几秒钟过后···</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">爬虫：美女，有票吗~~~？</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">12306：<span style="color: black;">无</span>，滚！</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">几秒钟过后···</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">爬虫：美女，有票吗~~~？</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">12306：<span style="color: black;">无</span>！！滚啊！！！</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">······</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">而<span style="color: black;">这般</span>的搭讪，春运的时候一天要<span style="color: black;">出现</span>1500亿次。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">12306：老娘真难。</span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/471dc65f1aae45548ab907eaefb2f857~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725101967&amp;x-signature=qdWvqueReI8oOJpxSw8T3X%2Fiw6s%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">亦</span>正是这个<span style="color: black;">原由</span>，12306的服务器有着近乎恐怖的抗压能力。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">每年双十一的时候，阿里巴巴都如临大敌，但淘宝双十一上亿的<span style="color: black;">拜访</span>量跟12306的1500亿比起来还是差了<span style="color: black;">非常多</span>。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">况且</span>每年双十一就一天，春运却有整整<span style="color: black;">一月</span>！</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">12306：老娘太难了！！</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">即便如此，每年双十一刚<span style="color: black;">起始</span>的时候，淘宝偶尔还是会<span style="color: black;">显现</span>卡顿、断网的现象。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">事实上，<span style="color: black;">针对</span>绝大<span style="color: black;">都数</span>网站和APP<span style="color: black;">来讲</span>，能够<span style="color: black;">同期</span><span style="color: black;">保准</span>上百万用户的无压力<span style="color: black;">运用</span>就<span style="color: black;">已然</span>很不错了，上亿<span style="color: black;">拜访</span>量的<span style="color: black;">状况</span>基本上都不存在。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">因此呢</span>爬虫的爬取功能不仅<span style="color: black;">能够</span>获取数据，还<span style="color: black;">能够</span><span style="color: black;">做为</span>一种攻击方式，让一个网站在<span style="color: black;">短期</span>内<span style="color: black;">拜访</span>量激增，<span style="color: black;">从而</span><span style="color: black;">引起</span>网站崩溃。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">因此</span>人们不得不制订了<span style="color: black;">关联</span>的法律法规，限制人们对爬虫的<span style="color: black;">运用</span>。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">而<span style="color: black;">咱们</span>在登录网站时输入的<span style="color: black;">各样</span>各样的奇葩验证码，在某些方面<span style="color: black;">亦</span>是为了<span style="color: black;">增多</span>爬虫<span style="color: black;">拜访</span>网站的难度。</span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/c464f5f032274b85b7a0e238bcedf7ca~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725101967&amp;x-signature=IgslThCpRuQUZ2rHg3dOD5ZAiQg%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">可是再严格的规范，<span style="color: black;">亦</span>防不住有些头铁的犯罪分子，以及···技术小白。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">早在<span style="color: black;">博主</span>还在读书的时候，就<span style="color: black;">已然</span>听过无数的悲剧。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">非常多</span>小白刚学会编写爬虫，就兴致勃勃地丢到了网络上准备试一试水。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">然而有些人在<span style="color: black;">研发</span>的时候却忘了给爬虫设定一个<span style="color: black;">拜访</span>频率，结果爬虫顿时化身加特林菩萨，以毫秒为单位疯狂<span style="color: black;">拜访</span><span style="color: black;">目的</span>网站，对<span style="color: black;">目的</span>网站进行了一场<span style="color: black;">理学</span>超度。</span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/295173fb6c3b4d33939b14ce59a97d34~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725101967&amp;x-signature=2fj1vBblo6NKkDXtXygZTPmgegU%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">最后</span>的<span style="color: black;">结果</span>除了<span style="color: black;">目的</span>网站崩溃，还有<span style="color: black;">便是</span>看守所里又多了一个秃头程序员。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;"><span style="color: black;">亦</span>许正是<span style="color: black;">由于</span>这些悲剧太多吧，<span style="color: black;">博主</span>的下一届学弟学妹们就多了一堂课——信息安全法律<span style="color: black;">基本</span>···</span></span></p>
    <div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p6-sign.toutiaoimg.com/tos-cn-i-qvj2lq49k0/f16f9a413f0047f2b9ebba8b18412432~noop.image?_iz=58558&amp;from=article.pc_detail&amp;lk3s=953192f4&amp;x-expires=1725101967&amp;x-signature=%2Bucj2zxTF1bhLXkCqJiTbdgNPBM%3D" style="width: 50%; margin-bottom: 20px;"></div>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">随着人工智能的发展，爬虫识别白百合和王珞丹的能力可能比人类还要强，各大网站的验证码<span style="color: black;">亦</span>是<span style="color: black;">因此呢</span>越来越<span style="color: black;">繁杂</span>。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">但其实<span style="color: black;">每一个</span>技术诞生之初都是无罪的，无奈有些人拿着这些技术去做<span style="color: black;">有些</span><span style="color: black;">违法</span>的勾当，使得技术本身<span style="color: black;">亦</span><span style="color: black;">再也不</span>纯洁。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">而对技术的污染，最后的恶果还是得由<span style="color: black;">咱们</span>自己吃下。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">12306不得不耗费<span style="color: black;">更加多</span>的精力<span style="color: black;">研发</span>新技术来屏蔽抢票软件，并且自己<span style="color: black;">研发</span>出了候补购票的功能，来满足<span style="color: black;">大众</span>的抢票<span style="color: black;">需要</span>。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">各大网站的<span style="color: black;">研发</span>人员<span style="color: black;">亦</span>不得不耗费<span style="color: black;">海量</span>精力<span style="color: black;">研发</span><span style="color: black;">各样</span>新奇的验证码，而用户<span style="color: black;">亦</span>不得不花费<span style="color: black;">更加多</span>的时间去解锁验证码。</span></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">而这种技术带来的便利和痛苦并存的状态，将<span style="color: black;">始终</span><span style="color: black;">伴同</span>着人类文明···</span></span></p>

nqkk58 发表于 2024-10-8 10:42:32

我赞同你的看法，你的智慧让人佩服，谢谢分享。

m5k1umn 发表于 2024-11-13 11:28:47

回顾历史，我们不难发现：无数先辈用鲜血和生命铺就了中华民族复兴的康庄大道。

wrjc1hod 发表于 7 天前

对于这个问题，我有不同的看法...

查看完整版本: 什么是“爬虫”？