影刀RPA实操指南Boss直聘猎聘招聘数据自动采集与人才库管理系统痛点切入一个HR每天打开Boss直聘、猎聘、智联三个平台手动翻看简历、复制粘贴候选人信息到Excel、逐个发消息约面试。更不用说还要跟踪招聘进度、汇总渠道效果、更新人才库。这套流程极其低效一个岗位少说要花2小时多个岗位并行则全天陷在重复操作里。招聘平台还有严格的反爬策略——频繁操作触发滑块验证码、IP被限流、元素选择器频繁变动。本文将用影刀RPA构建一套招聘数据自动采集与人才库管理系统把所有手工操作压缩到一键完成。核心方法1. Boss直聘元素定位策略店群矩阵自动化突破运营极限Boss直聘的页面结构经过特殊设计传统的CSS选择器和XPath经常失效。关键在于理解其React渲染机制和动态属性命名规律 Boss直聘简历采集模块 注意Boss直聘使用Reactclass名包含hash需要基于语义定位 importtimefromseleniumimportwebdriverfromselenium.webdriver.common.byimportByfromselenium.webdriver.support.uiimportWebDriverWaitfromselenium.webdriver.supportimportexpected_conditionsasECfromselenium.webdriver.common.action_chainsimportActionChainsimportpandasaspdimportrandomclassBossZhipinBot:Boss直聘自动化采集def__init__(self):# 反检测配置——模拟正常用户行为optionswebdriver.ChromeOptions()options.add_argument(--disable-blink-featuresAutomationControlled)options.add_experimental_option(excludeSwitches,[enable-automation])options.add_experimental_option(useAutomationExtension,False)# 关闭webdriver标志options.add_argument(--disable-infobars)# 随机窗口大小避免指纹识别options.add_argument(f--window-size{random.randint(1360,1400)},{random.randint(768,900)})self.driverwebdriver.Chrome(optionsoptions)# 注入JS隐藏webdriver特征self.driver.execute_cdp_cmd(Page.addScriptToEvaluateOnNewDocument,{source: Object.defineProperty(navigator, webdriver, {get: () undefined}); Object.defineProperty(navigator, plugins, {get: () [1,2,3,4,5]}); Object.defineProperty(navigator, languages, {get: () [zh-CN,zh,en]}); })self.waitWebDriverWait(self.driver,10)deflogin_boss(self,cookie_strNone):登录Boss直聘——推荐使用Cookie注入避免验证码self.driver.get(https://www.zhipin.com/web/chat/index)ifcookie_str:# Cookie注入方式登录避免滑块验证码forcookie_itemincookie_str.split(; ):name,valuecookie_item.split(,1)self.driver.add_cookie({name:name,value:value,domain:.zhipin.com})self.driver.refresh()# 等待聊天列表加载完成——这是登录成功的关键判断标志self.wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,.chat-list)))print(登录成功)defsearch_candidates(self,keyword,city_code101010100,experience103): 搜索候选人 keyword: 搜索关键词如Python city_code: 城市编码101010100北京 experience: 经验要求1033-5年 # 构造搜索URL——Boss直聘使用URL参数传递过滤条件url(fhttps://www.zhipin.com/web/geek/jobf?query{keyword}city{city_code}experience{experience})self.driver.get(url)time.sleep(random.uniform(2,4))# 随机延迟模拟人类浏览# 等待候选人列表加载——使用.text()包含文本的元素定位self.wait.until(EC.presence_of_element_located((By.XPATH,//div[contains(class,search-job-result)])))defparse_candidate_list(self,max_pages5): 解析候选人列表页面 返回结构化数据 candidates[]forpageinrange(max_pages):print(f正在采集第{page1}页...)# 获取当前页所有候选人卡片# 注意Boss直聘的card类名包含动态hash使用starts-with匹配cardsself.driver.find_elements(By.XPATH,//div[starts-with(class,candidate-card)])forcardincards:try:candidate{# 姓名——使用//text()精确定位name:card.find_element(By.XPATH,.//span[contains(class,name)]).text,# 工作年限experience:card.find_element(By.XPATH,.//p[contains(text(),经验)]).text,# 学历education:card.find_element(By.XPATH,.//p[contains(text(),学历)]).text,# 期望职位expect_job:card.find_element(By.XPATH,.//span[contains(class,expect-position)]).text,# 期望薪资expect_salary:card.find_element(By.XPATH,.//span[contains(class,salary)]).text,# 当前状态——如在职-月内到岗status:card.find_element(By.XPATH,.//span[contains(class,status)]).textifcard.find_elements(By.XPATH,.//span[contains(class,status)])else未知,# 采集时间戳collect_time:time.strftime(%Y-%m-%d %H:%M:%S)}candidates.append(candidate)exceptExceptionase:print(f解析卡片异常:{e})continue# 翻页前的随机延迟time.sleep(random.uniform(3,6))# 翻页逻辑——Boss直聘使用点击加载更多try:next_btnself.driver.find_element(By.XPATH,//div[contains(class,pagination)]//a[contains(class,next)])ifdisabledinnext_btn.get_attribute(class):print(已到最后一页)breaknext_btn.click()except:print(翻页失败可能已到最后一页或触发验证)breakreturncandidatesdefsave_to_excel(self,data,filename):保存到Exceldfpd.DataFrame(data)# 去重——同一候选人可能多次出现dfdf.drop_duplicates(subset[name])df.to_excel(filename,indexFalse)print(f已保存{len(df)}条数据到{filename})2. 猎聘高级搜索策略猎聘的搜索接口更丰富支持学历、薪资、行业、公司规模等多维度过滤但页面结构更复杂classLiepinBot:猎聘自动化采集def__init__(self):optionswebdriver.ChromeOptions()options.add_argument(--disable-blink-featuresAutomationControlled)self.driverwebdriver.Chrome(optionsoptions)self.waitWebDriverWait(self.driver,15)defadvanced_search(self,params): 猎聘高级搜索 params { keyword: 产品经理, city: 010, # 北京 salary: 15*25, # 15k-25k education: 4, # 本科 work_year: 3, # 3-5年 industry: 020 # 互联网/电商 } # 猎聘搜索URL参数比Boss直聘更多base_urlhttps://www.liepin.com/zhaopin/query_parts[]# 拼接搜索参数——猎聘使用dqs参数传递多维过滤条件dqs_parts[]ifcityinparams:dqs_parts.append(f010_{params[city]})# 工作地点ifsalaryinparams:dqs_parts.append(f020_{params[salary]})# 薪资范围ifeducationinparams:dqs_parts.append(f030_{params[education]})# 学历ifwork_yearinparams:dqs_parts.append(f040_{params[work_year]})# 工作年限dqs|.join(dqs_parts)urlf{base_url}?key{params[keyword]}dqs{dqs}self.driver.get(url)time.sleep(random.uniform(3,5))defparse_job_list(self):解析职位列表jobs[]# 猎聘使用li元素包裹每个职位job_itemsself.driver.find_elements(By.XPATH,//ul[contains(class,job-list-box)]/li)forjobinjob_items:try:job_data{title:job.find_element(By.XPATH,.//h3).text,company:job.find_element(By.XPATH,.//p[contains(class,company-name)]).text,salary:job.find_element(By.XPATH,.//span[contains(class,salary)]).text,location:job.find_element(By.XPATH,.//span[contains(class,area)]).text,# 福利标签——猎聘特有tags:,.join([tag.textfortaginjob.find_elements(By.XPATH,.//div[contains(class,tag-list)]/span)]),# HR活跃度——猎聘特色指标hr_active:job.find_element(By.XPATH,.//span[contains(class,hr-active)]).textifjob.find_elements(By.XPATH,.//span[contains(class,hr-active)])else,publish_time:job.find_element(By.XPATH,.//span[contains(class,time)]).text}jobs.append(job_data)exceptExceptionase:print(f解析职位异常:{e})continuereturnjobs3. 人才库管理系统采集的数据需要统一存储和管理下面构建一个基于SQLite的轻量级人才库importsqlite3fromdatetimeimportdatetime,timedeltaclassTalentPool:人才库管理系统def__init__(self,db_pathtalent_pool.db):self.connsqlite3.connect(db_path)self._init_tables()def_init_tables(self):初始化数据库表结构cursorself.conn.cursor()# 候选人主表cursor.execute( CREATE TABLE IF NOT EXISTS candidates ( id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, phone TEXT, email TEXT, current_company TEXT, current_position TEXT, experience_years REAL, education TEXT, expected_salary TEXT, skills TEXT, source_platform TEXT, -- 来源boss/liepin/zhilian source_url TEXT, -- 简历原始链接 status TEXT DEFAULT new, -- 状态new/contacted/interviewing/offered/hired/rejected tag TEXT, -- 自定义标签 notes TEXT, -- 备注 resume_file TEXT, -- 简历文件路径 created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ) )# 沟通记录表cursor.execute( CREATE TABLE IF NOT EXISTS communications ( id INTEGER PRIMARY KEY AUTOINCREMENT, candidate_id INTEGER, method TEXT, -- 沟通方式online_chat/phone/email content TEXT, result TEXT, -- 结果interested/not_interested/pending created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, FOREIGN KEY (candidate_id) REFERENCES candidates(id) ) )# 面试记录表cursor.execute( CREATE TABLE IF NOT EXISTS interviews ( id INTEGER PRIMARY KEY AUTOINCREMENT, candidate_id INTEGER, interview_round INTEGER, -- 第几轮面试 interview_type TEXT, -- 类型phone/video/onsite scheduled_time TIMESTAMP, interviewer TEXT, feedback TEXT, result TEXT, -- pass/fail/pending created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, FOREIGN KEY (candidate_id) REFERENCES candidates(id) ) )# 招聘渠道效果统计表cursor.execute( CREATE TABLE IF NOT EXISTS channel_stats ( id INTEGER PRIMARY KEY AUTOINCREMENT, channel_name TEXT, -- 渠道名称 date DATE, job_title TEXT, resume_count INTEGER DEFAULT 0, qualified_count INTEGER DEFAULT 0, interview_count INTEGER DEFAULT 0, offer_count INTEGER DEFAULT 0, hire_count INTEGER DEFAULT 0, cost REAL DEFAULT 0, -- 渠道费用 created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ) )self.conn.commit()defimport_candidates(self,candidates_list):批量导入候选人去重cursorself.conn.cursor()inserted0skipped0forcincandidates_list:# 根据姓名来源平台去重cursor.execute(SELECT id FROM candidates WHERE name? AND source_platform?,(c.get(name),c.get(source_platform)))ifcursor.fetchone():skipped1continuecursor.execute( INSERT INTO candidates (name, current_position, experience_years, education, expected_salary, skills, source_platform, source_url, status) VALUES (?, ?, ?, ?, ?, ?, ?, ?, new) ,(c.get(name),c.get(expect_job),c.get(experience),c.get(education),c.get(expect_salary),c.get(skills,),c.get(source_platform),c.get(source_url,),))inserted1self.conn.commit()print(f导入完成新增{inserted}人跳过{skipped}人已存在)defget_pipeline_stats(self):获取招聘流程漏斗数据cursorself.conn.cursor()stats{}# 各阶段人数统计forstatusin[new,contacted,interviewing,offered,hired,rejected]:cursor.execute(SELECT COUNT(*) FROM candidates WHERE status?,(status,))stats[status]cursor.fetchone()[0]returnstatsdefget_channel_effectiveness(self,days30):统计各招聘渠道效果cursorself.conn.cursor()date_threshold(datetime.now()-timedelta(daysdays)).strftime(%Y-%m-%d)cursor.execute( SELECT source_platform, COUNT(*) as total, SUM(CASE WHEN statusinterviewing OR statusoffered OR statushired THEN 1 ELSE 0 END) as qualified, SUM(CASE WHEN statushired THEN 1 ELSE 0 END) as hired FROM candidates WHERE created_at ? GROUP BY source_platform ,(date_threshold,))resultscursor.fetchall()print(f\n 近{days}天招聘渠道效果 )print(f{渠道:12}{投递数:8}{进入面试:10}{入职数:8})forrowinresults:print(f{row[0]:12}{row[1]:8}{row[2]:10}{row[3]:8})returnresultsdefclose(self):self.conn.close()# 主流程示例 if__name____main__:# 初始化人才库poolTalentPool()# Boss直聘采集bossBossZhipinBot()boss.login_boss(cookie_stryour_cookie_here)boss.search_candidates(Python后端开发,city_code101010100)boss_candidatesboss.parse_candidate_list(max_pages3)# 标记来源平台forcinboss_candidates:c[source_platform]bosspool.import_candidates(boss_candidates)# 猎聘采集类似流程# liepin LiepinBot()# ...# 输出渠道效果统计pool.get_channel_effectiveness(days30)# 输出招聘漏斗statspool.get_pipeline_stats()print(f\n招聘漏斗:{stats})pool.close()boss.driver.quit()常见问题速查temu店群自动化报活动案例序号现象原因解决方式1Boss直聘页面打开后无法获取元素selenium报stale element referenceReact虚拟DOM重新渲染导致已定位元素失效每次操作前重新find_element不要保存元素引用使用WebDriverWait等待元素稳定后再操作2访问几次后被重定向到滑块验证码页面Boss直聘检测到自动化特征navigator.webdrivertrue注入CDP脚本覆盖navigator.webdriver属性关闭enable-automation开关随机化窗口尺寸3猎聘搜索结果数量与实际不符猎聘使用懒加载滚动到底部才加载更多使用driver.execute_script(window.scrollTo(0, document.body.scrollHeight))触底加载然后等2秒再解析4采集的薪资格式不统一有15K-25K也有15k-25k·14薪不同HR填写格式不统一使用正则统一提取数字部分re.findall(r(\d), salary_str)特殊处理·14薪中的年薪倍数5Cookie登录后页面自动跳转到登录页Cookie已过期或domain设置错误Cookie有效期通常24小时每天需要重新获取domain必须设置为.zhipin.com注意前面的点6采集数据中姓名显示为张先生/李女士而非全名Boss直聘在搜索结果中隐藏了全名点击进入详情页才有需要二级爬取搜索结果只取基本信息点击卡片进入详情页获取完整信息7SQLite写入大量数据时锁表多线程同时写入SQLiteSQLite不支持高并发写改为单线程写入或使用WAL模式PRAGMA journal_modeWAL;推荐资源Boss直聘开放平台https://open.zhipin.com猎聘企业版APIhttps://ent.liepin.comSelenium反检测指南https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealthSQLite WAL模式文档https://www.sqlite.org/wal.html内容标签#影刀RPA#Boss直聘#猎聘#招聘自动化#人才库#反爬策略#元素定位#数据采集作者林焱