摘要
该设计在实现多个抓取线程管理和利用线程池发送抓取网页的同时,采用非递归爬行算法,利用Java多线程技术和基于内存的作业队列来增加、分配和处理运行过程中的URL.搜索引擎简易客户端的设计最终采用JSP(Java Server Pages)技术完成.
Abstract
This design uses non recursive crawling algorithm,uses Java multithreading technology and memory based job queue to add,allocate and process URL in the operation process,and uses thread pool to realize the management of multiple fetching threads,and concurrently fetches web pages.Finally,JSP(Java Server Pages)technology is used to complete the design of simple search engine client.
NETL
NSTL
万方数据