目 录摘 要 IAbstract III1绪论 11.1选题背景及意义 11.2研究现状及趋势 11.3研究主要内容 22相关技术简介 32.1开发工具 32.1.1 JDK1.7 32.1.2 eclipse luna 32.1.3 Hadoop 2.7.2 32.1.4 hbase 1.1.3 32.1.5 hive 1.2.1 32.1.6 zookeeper 3.4.8 42.1.7 mysql 5.5 42.1.8 swing 42.1.9 VMware Workstation 12 Pro 42.1.10其他辅助开发工具包 52.2开发环境搭建 62.2.1 三台Linux虚拟机的搭建 62.2.2 配置静态ip 72.2.3 SSH安装及免密码登陆 82.2.4修改hosts文件 92.2.5安装jdk 102.2.6 Hadoop安装与配置 112.2.7 Zookeeper安装与配置 132.2.8 Hbase的安装与配置 154.2.9 Hive的安装与配置 162.2.10 mysql的安装与配置 174.2.11 Hive与Hbase的整合 192.2.12 Hadoop-eclipse-plugin插件安装 192.2.13 WindowBuilder插件 193 需求分析 213.1可行性分析 213.1.1可行性研究介绍 213.1.2可行性分析 213.2需求分析 213.2.1需求分析作用 213.2.2功能需求 224 可视化设计 234.1界面设计 234.1.1前台界面设计 234.2模块划分 244.2.1用户注册与登录模块 244.2.2上传气象数据模块 244.2.3数据分析模块 254.2.4定时更新气象数据模块 254.2.5历史气象数据查询模块 254.2.6今日气象数据动态显示模块 274.2.7历史气象走势与极端气象查询模块 274.2.8数据库设计模块 274.3数据库设计E-R图 275 可视化界面 315.1界面实现 315.1.1主窗体的创建与总体布局 315.1.2 界面主要控件实现 325.1.3界面动态图形展示实现 335.2各功能模块实现 355.2.1用户注册与登录模块实现 355.2.2上传数据模块实现 375.2.3数据分析模块实现 385.2.4定时在线更新与分析模块实现 395.2.5历史气象数据查询模块实现 415.2.6今日气象数据动态显示模块实现 425.2.7历史气象走势模块实现 425.2.8数据库模块实现 426 可视化展示 456.1用户注册与登录功能展示 456.1.1用户注册 456.1.2用户登录 456.2上传数据功能展示 466.3数据分析与定时更新功能展示 476.3.1数据分析 476.3.2定时更新 486.4数据库内容展示 496.5今日气象展示 496.6历史气象展示 506.7历史气象走势展示 537 总结与展望 557.1总结 557.2未来展望 55参考文献 56致 谢 594 可视化设计本系统大体上分为前台界面和后台Hadoop集群两部分。通过将系统的前台与后台划分开来使本来看起来复杂庞大的可视化设计简单清晰了很多便于开发时根据开发的部分适时调整自己的定位与开发方法。4.1界面设计4.1.1前台界面设计前台主要界面有1用户注册界面提供用户名密码二次输入密码等用户注册的基本功能。2用户登录界面提供用户名密码管理员权限输入等功能。3管理员界面分为以下6个部分1历史气象数据查询界面提供查询历史信息的基本界面如选择城市时间。也提供了气象数据的查询方式如查询当天气象情况当月情况当年情况等。查询结果以图形的形式展现。2今日气象数据查询界面展现全国各省会今日的气象概况展现形式参考气象预报提供以中国地图为背景动态展示气象情况的效果3上传气象数据界面提供选择城市选择文件点击上传等功能。4数据分析界面提供选择时间范围选择分析数据方式如客户端分析Hive分析等功能。5定时数据更新界面提供定时获取数据与分析数据的详细日志进行界面展现的功能。6历史气象变迁查询界面提供选择指定城市展现选定时间范围内气象情况的走势情况展现形式为折线图。4普通用户界面分为以下三个部分1历史气象数据查询界面同管理员界面2今日气象数据查询界面同管理员界面3历史气象变迁及恶劣气象情况查询界面同管理员界面4.2模块划分将系统模块化使我们的设计思路更加清晰层次分明。在详细设计时可以通过将模块分别开发来提高开发效率也便于单元测试保证程序的稳定性。本系统主要分为八大模块1用户注册与登录模块2上传气象数据模块3数据分析模块4定时更新气象数据模块5历史气象数据查询模块6今日气象数据动态显示模块7历史气象变迁模块8数据库设计模块4.2.1用户注册与登录模块模块主要功能点击注册按钮进入到用户注册界面输入用户名和密码等信息即可完成注册注册成功之后将用户信息保存到数据库中其中加入不符合条件的用户信息的前台简单判定1输入判定1两次输入密码不相同2用户名为空3密码为空2用户信息判定1用户注册用户名是否存在的判定2用户登录用户名密码是否正确以上不符合条件的注册与登录信息皆会以弹窗的形式告知用户4.2.2上传气象数据模块模块主要功能1从指定的气象数据网站上下载气象数据文件点击选择城市与选择文件按钮从本地选择下载好的数据进行上传。2将待上传的数据文件进行信息提取气象风速云量气象总体概况等。3将提取后的文件上传到Hbase。4将源数据文件上传到HDFS以备文件里的其他信息用于以后的功能扩展。4.2.3数据分析模块模块主要功能本文转载自http://www.biyezuopin.vip/onews.asp?id14114数据分析模块提供选择时间范围使用客户端分析或Hive分析Hbase所保存气象数据的功能。分析详细如下1分析每天全国各地最高/最低气象最高/最低风速所在的城市2分析全国各城市每个月的气象情况如当月平均气象最高气象最高风速情况等。4.2.4定时更新气象数据模块模块主要功能如下1利用一些气象数据网站提供的API在线获取信息将这些信息保存到Hbase.每天定时更新方便快捷便于城市气象数据的实时获取与维护。2定时分析当天获取到的气象数据分析内容参照数据分析模块并将分析进度展现出来。/* SQLyog Ultimate v12.08 (64 bit) MySQL - 5.7.23-log : Database - userdb ********************************************************************* */ /*!40101 SET NAMES utf8 */; /*!40101 SET SQL_MODE*/; /*!40014 SET OLD_UNIQUE_CHECKSUNIQUE_CHECKS, UNIQUE_CHECKS0 */; /*!40014 SET OLD_FOREIGN_KEY_CHECKSFOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS0 */; /*!40101 SET OLD_SQL_MODESQL_MODE, SQL_MODENO_AUTO_VALUE_ON_ZERO */; /*!40111 SET OLD_SQL_NOTESSQL_NOTES, SQL_NOTES0 */; CREATE DATABASE /*!32312 IF NOT EXISTS*/userdb /*!40100 DEFAULT CHARACTER SET utf8 */; USE userdb; /*Table structure for table temperature */ DROP TABLE IF EXISTS temperature; CREATE TABLE temperature ( Tem_Date varchar(10) NOT NULL, Tem_Max double DEFAULT NULL, Tem_Min double DEFAULT NULL, Tem_Avg double DEFAULT NULL, PRIMARY KEY (Tem_Date) ) ENGINEInnoDB DEFAULT CHARSETutf8; /*Data for the table temperature */ insert into temperature(Tem_Date,Tem_Max,Tem_Min,Tem_Avg) values (01/01,-58,-362,-169.7674418604651),(01/02,-85,-387,-185.56944444444446),(01/03,-51,-391,-169.47222222222223),(01/04,-47,-375,-172),(01/05,-89,-368,-186.76279069767443),(01/06,-66,-378,-202.5720930232558),(01/07,-77,-392,-188.73953488372092),(01/08,-47,-363,-156.99065420560748),(01/09,-7,-327,-123.66976744186047),(01/10,-15,-338,-141.7906976744186),(01/11,10,-314,-144.75),(01/12,-23,-308,-147.8186046511628),(01/13,-37,-325,-155.6435185185185),(01/14,-42,-374,-150.67906976744186),(01/15,-37,-375,-167.41860465116278),(01/16,-34,-392,-185.6898148148148),(01/17,-60,-398,-206.0186046511628),(01/18,-68,-357,-179.8139534883721),(01/19,-31,-343,-156.11162790697674),(01/20,-20,-336,-157.33644859813083),(01/21,-2,-342,-142.29302325581395),(01/22,15,-269,-122.38888888888889),(01/23,-44,-310,-150.85185185185185),(01/24,-49,-298,-137.49302325581397),(01/25,-39,-322,-136.52558139534884),(01/26,6,-343,-119.9766355140187),(01/27,19,-295,-116.53917050691244),(01/28,-32,-317,-141.36320754716982),(01/29,21,-324,-122.05581395348837),(01/30,-9,-379,-158.16589861751152),(01/31,-59,-393,-172.56221198156683),(02/01,-19,-399,-167.89814814814815),(02/02,20,-354,-155.94495412844037),(02/03,-2,-407,-186.77064220183487),(02/04,-106,-420,-214.9813953488372),(02/05,-116,-428,-231.19444444444446),(02/06,-67,-437,-235.16589861751152),(02/07,-141,-387,-227.09259259259258),(02/08,-97,-358,-197.75813953488372),(02/09,-76,-380,-185.89767441860465),(02/10,-71,-394,-183.94907407407408),(02/11,-64,-380,-190.72222222222223),(02/12,-45,-374,-168.85581395348837),(02/13,-53,-369,-155.90232558139536),(02/14,-31,-360,-145.86511627906978),(02/15,-36,-344,-135.94930875576037),(02/16,8,-280,-97.75813953488372),(02/17,44,-295,-80.45161290322581),(02/18,62,-268,-63.646511627906975),(02/19,41,-275,-74.22325581395349),(02/20,47,-254,-64.30232558139535),(02/21,72,-237,-50.43255813953488),(02/22,90,-234,-47.972093023255816),(02/23,109,-199,-19.07906976744186),(02/24,68,-274,-55.87906976744186),(02/25,41,-282,-73.75462962962963),(02/26,57,-245,-60.585253456221196),(02/27,89,-218,-41.53488372093023),(02/28,84,-254,-44.055813953488375),(03/01,88,-228,-34.265116279069765),(03/02,124,-204,-16.455813953488374),(03/03,108,-209,-22.85185185185185),(03/04,113,-199,-10.744186046511627),(03/05,107,-254,-35.6221198156682),(03/06,52,-243,-43.73953488372093),(03/07,93,-187,-22.888372093023257),(03/08,153,-204,12.586046511627908),(03/09,79,-209,-35.61574074074074),(03/10,92,-155,-7.268867924528302),(03/11,97,-207,-26.50232558139535),(03/12,56,-240,-56.31018518518518),(03/13,42,-225,-48.92631578947368),(03/14,61,-240,-43.42325581395349),(03/15,43,-179,-42.300925925925924),(03/16,73,-184,-22.057894736842105),(03/17,156,-135,28.416666666666668),(03/18,182,-116,36.976851851851855),(03/19,181,-209,10.221198156682028),(03/20,78,-234,-38.273148148148145),(03/21,70,-236,-64.31506849315069),(03/22,105,-258,-54.252252252252255),(03/23,121,-175,-29.65765765765766),(03/24,135,-139,-0.8687782805429864),(03/25,122,-218,-17.368663594470046),(03/26,106,-270,-37.0462962962963),(03/27,112,-243,-35.81481481481482),(03/28,135,-247,-33.61395348837209),(03/29,161,-223,-22.046511627906977),(03/30,196,-214,-18.46330275229358),(03/31,163,-160,-19.28110599078341),(04/01,181,-220,-11.175115207373272),(04/02,189,-147,17.792626728110598),(04/03,212,-92,50.14418604651163),(04/04,210,-119,35.2),(04/05,205,-112,24.409302325581397),(04/06,156,-115,26.02777777777778),(04/07,110,-102,15.790697674418604),(04/08,116,-157,22.232558139534884),(04/09,145,-152,37.55348837209302),(04/10,167,-136,44.78604651162791),(04/11,175,-110,66.69302325581396),(04/12,200,-139,87.05581395348837),(04/13,202,-145,43.02325581395349),(04/14,193,-109,54.70048309178744),(04/15,234,-20,117.31162790697674),(04/16,276,-110,119.69907407407408),(04/17,245,-147,60.95813953488372),(04/18,173,-162,24.11627906976744),(04/19,208,-70,59),(04/20,181,-130,65.23720930232558),(04/21,180,-129,64.61395348837209),(04/22,261,-64,109.1767441860465),(04/23,251,-55,89.55760368663594),(04/24,190,-152,56.717592592592595),(04/25,162,-142,62.60648148148148),(04/26,183,-50,88.46046511627907),(04/27,229,-42,124.24651162790698),(04/28,228,-72,124.32093023255814),(04/29,225,18,128.84722222222223),(04/30,223,-77,112.23720930232558),(05/01,231,-55,92.32558139534883),(05/02,269,-19,125.94907407407408),(05/03,285,-7,155.7953488372093),(05/04,295,-104,108.25345622119816),(05/05,207,-80,62.39252336448598),(05/06,193,-22,101.39814814814815),(05/07,247,14,135.08333333333334),(05/08,246,-75,116.73953488372094),(05/09,272,0,139.36574074074073),(05/10,319,-29,161.50232558139535),(05/11,306,-12,155.10185185185185),(05/12,282,10,162.89252336448598),(05/13,266,7,118.07870370370371),(05/14,242,-32,113.36574074074075),(05/15,278,-14,155.25116279069766),(05/16,260,0,130.6435185185185),(05/17,259,2,143.65581395348838),(05/18,268,-5,123.56744186046511),(05/19,226,25,120.44651162790697),(05/20,247,45,107.09767441860465),(05/21,295,-57,110.2046511627907),(05/22,286,12,120.97183098591549),(05/23,350,57,176.51322751322752),(05/24,374,20,213.5305164319249),(05/25,328,-9,173.86511627906978),(05/26,300,-25,140.0139534883721),(05/27,218,-39,127.14953271028037),(05/28,229,-20,125.51869158878505),(05/29,269,15,148.906103286385),(05/30,285,-15,137.87906976744185),(05/31,252,-12,140.26699029126215),(06/01,266,-20,154.3860465116279),(06/02,276,6,162.76279069767443),(06/03,275,30,174.34418604651162),(06/04,277,21,187.06976744186048),(06/05,275,2,176.04651162790697),(06/06,307,37,185.53488372093022),(06/07,303,50,180.05581395348838),(06/08,299,17,189.33953488372094),(06/09,282,50,186.41203703703704),(06/10,306,75,196.99537037037038),(06/11,317,57,197.7813953488372),(06/12,302,70,200.3953488372093),(06/13,270,72,170.12037037037038),(06/14,258,1,141.3778801843318),(06/15,270,15,153.9181818181818),(06/16,282,26,171.7590909090909),(06/17,280,26,175.03240740740742),(06/18,325,85,196.89814814814815),(06/19,342,30,195.25925925925927),(06/20,315,40,176.70697674418605),(06/21,315,70,202.31627906976743),(06/22,331,107,218.27649769585253),(06/23,363,132,235.53917050691246),(06/24,340,96,235.9212962962963),(06/25,323,70,215.91627906976746),(06/26,301,75,197.6712962962963),(06/27,315,16,171.61111111111111),(06/28,328,12,163.25581395348837),(06/29,340,46,172.03255813953487),(06/30,327,35,184.8119266055046),(07/01,296,85,197.12442396313364),(07/02,305,60,191.65898617511522),(07/03,316,80,197.82027649769586),(07/04,320,20,201.0091743119266),(07/05,300,32,181.4884792626728),(07/06,311,67,202.3440366972477),(07/07,340,62,226.98617511520737),(07/08,327,107,241.14883720930231),(07/09,342,107,244.17592592592592),(07/10,326,102,227.21495327102804),(07/11,306,96,214.03773584905662),(07/12,313,105,215.04651162790697),(07/13,315,97,218.62325581395348),(07/14,328,97,218.3674418604651),(07/15,335,87,225.3860465116279),(07/16,316,90,217.353488372093),(07/17,331,110,224.46511627906978),(07/18,335,106,233.6),(07/19,326,168,243.1813953488372),(07/20,329,142,243.28372093023256),(07/21,321,120,230.01869158878506),(07/22,330,106,221.58139534883722),(07/23,307,122,212.53023255813955),(07/24,299,102,189.70697674418605),(07/25,280,65,186.75348837209302),(07/26,327,57,198.32407407407408),(07/27,302,128,221.2),(07/28,327,101,224.5906976744186),(07/29,322,120,237.2232558139535),(07/30,327,112,218.07906976744187),(07/31,316,100,220.2073732718894),(08/01,322,92,223.66511627906976),(08/02,337,115,235.0139534883721),(08/03,347,96,238.7314814814815),(08/04,370,86,247.80751173708921),(08/05,327,41,228.87962962962962),(08/06,310,75,202.6),(08/07,307,76,194.90277777777777),(08/08,325,82,190.0648148148148),(08/09,347,86,182.36915887850466),(08/10,257,100,173.88679245283018),(08/11,261,57,179.1860465116279),(08/12,295,75,194.12962962962962),(08/13,332,66,208.6139534883721),(08/14,345,37,209.37383177570092),(08/15,318,72,203.90232558139536),(08/16,267,86,180.67906976744186),(08/17,267,65,175.26728110599078),(08/18,246,37,168.44131455399062),(08/19,265,67,180.3287037037037),(08/20,282,115,183.52558139534884),(08/21,263,106,174.96744186046513),(08/22,297,90,180.58256880733944),(08/23,250,82,172.8479262672811),(08/24,254,75,174.86976744186046),(08/25,266,72,180.28240740740742),(08/26,287,82,185.53953488372093),(08/27,315,32,179.93119266055047),(08/28,332,45,175.28310502283105),(08/29,347,36,177.52534562211983),(08/30,347,35,186.38947368421051),(08/31,296,62,180.5462962962963),(09/01,302,-4,171.9953488372093),(09/02,280,55,177.3860465116279),(09/03,317,90,193.5860465116279),(09/04,328,106,212.08837209302325),(09/05,312,86,204.2325581395349),(09/06,322,87,207.74418604651163),(09/07,302,84,206.1860465116279),(09/08,279,17,172.7953488372093),(09/09,254,0,138.07943925233644),(09/10,225,-25,122.87441860465117),(09/11,282,20,145.5185185185185),(09/12,285,51,165.35813953488372),(09/13,227,-20,139.85340314136127),(09/14,247,-7,144.4),(09/15,256,-69,119.94418604651163),(09/16,213,-16,109.8232558139535),(09/17,222,-37,93.73148148148148),(09/18,217,-57,82.77251184834124),(09/19,257,-6,105.20370370370371),(09/20,267,-77,113.09302325581395),(09/21,287,-45,111.24537037037037),(09/22,296,-20,151.56744186046512),(09/23,311,-80,143.8186046511628),(09/24,307,-62,116.37209302325581),(09/25,300,-37,128.86976744186046),(09/26,300,6,152.62325581395348),(09/27,305,41,189.34562211981566),(09/28,276,-72,158.5767441860465),(09/29,240,-40,106.46261682242991),(09/30,290,-14,127.04608294930875),(10/01,298,-18,168.26388888888889),(10/02,284,-77,114.55440414507773),(10/03,150,-130,48.93023255813954),(10/04,165,-117,31.049180327868854),(10/05,190,-92,67.71627906976744),(10/06,216,-35,104.00925925925925),(10/07,236,-79,86.10232558139535),(10/08,237,-37,69.86976744186046),(10/09,229,-69,106.5925925925926),(10/10,221,-109,54.398148148148145),(10/11,210,-193,16.083333333333332),(10/12,189,-197,-1.6775700934579438),(10/13,189,-194,-0.33796296296296297),(10/14,197,-125,19.875),(10/15,192,-125,46.254629629629626),(10/16,169,-94,46.138888888888886),(10/17,180,-92,63.566820276497694),(10/18,181,-84,65.29439252336448),(10/19,189,-70,72.48387096774194),(10/20,192,-64,73.66197183098592),(10/21,163,-72,42.97674418604651),(10/22,191,-115,36.14883720930233),(10/23,192,-111,35.009049773755656),(10/24,184,-130,27.955156950672645),(10/25,156,-168,-2.939814814814815),(10/26,84,-150,-3.0324074074074074),(10/27,119,-96,12.89237668161435),(10/28,140,-134,26.219730941704036),(10/29,116,-152,9.991031390134529),(10/30,146,-122,41.300448430493276),(10/31,169,-144,32.860986547085204),(11/01,172,-195,15.702702702702704),(11/02,186,-237,-17.981900452488688),(11/03,102,-212,-24.800904977375566),(11/04,169,-125,7.745454545454545),(11/05,129,-228,-30.569506726457398),(11/06,30,-208,-61.593607305936075),(11/07,95,-238,-53.16279069767442),(11/08,73,-207,-53.13963963963964),(11/09,89,-189,-25.82511210762332),(11/10,101,-142,4.7207207207207205),(11/11,91,-243,-34.318385650224215),(11/12,36,-274,-72.79372197309416),(11/13,10,-326,-123.1447963800905),(11/14,26,-318,-143.34977578475338),(11/15,1,-319,-128.7072072072072),(11/16,-29,-290,-129.37387387387386),(11/17,-7,-273,-107.07239819004525),(11/18,-32,-202,-122.98206278026906),(11/19,18,-313,-142.76576576576576),(11/20,32,-311,-150.26696832579185),(11/21,47,-302,-111.1891891891892),(11/22,98,-254,-51.794520547945204),(11/23,83,-343,-104.60309278350516),(11/24,-35,-359,-151.07692307692307),(11/25,-40,-344,-130.46445497630333),(11/26,2,-344,-128.43093922651934),(11/27,-45,-345,-160.53365384615384),(11/28,-62,-342,-164.85096153846155),(11/29,-31,-333,-146.40865384615384),(11/30,-25,-294,-129.26923076923077),(12/01,-50,-313,-144.9047619047619),(12/02,-49,-307,-164.2095238095238),(12/03,-70,-323,-202.85849056603774),(12/04,-27,-387,-211.13888888888889),(12/05,7,-375,-191.7311320754717),(12/06,8,-350,-162.04651162790697),(12/07,-38,-372,-169.49302325581397),(12/08,-5,-374,-158.70232558139534),(12/09,-1,-334,-145.3440366972477),(12/10,5,-349,-153.97663551401868),(12/11,-16,-349,-174.69767441860466),(12/12,-11,-341,-144.4279069767442),(12/13,2,-323,-143.3813953488372),(12/14,-23,-334,-130.1627906976744),(12/15,7,-244,-115.40284360189574),(12/16,20,-354,-101.72093023255815),(12/17,-2,-379,-162.55348837209303),(12/18,-75,-392,-200),(12/19,-84,-393,-210.43867924528303),(12/20,-70,-399,-199.85581395348837),(12/21,-32,-405,-210.88837209302326),(12/22,-54,-408,-222.25233644859813),(12/23,-74,-405,-221.74883720930234),(12/24,-85,-399,-217.6602870813397),(12/25,-97,-402,-223.44600938967136),(12/26,-70,-400,-239.07246376811594),(12/27,-60,-399,-237.83091787439614),(12/28,-127,-410,-224.2378640776699),(12/29,-60,-417,-217.79),(12/30,-35,-415,-243.626213592233),(12/31,-35,-398,-227.03846153846155); /*!40101 SET SQL_MODEOLD_SQL_MODE */; /*!40014 SET FOREIGN_KEY_CHECKSOLD_FOREIGN_KEY_CHECKS */; /*!40014 SET UNIQUE_CHECKSOLD_UNIQUE_CHECKS */; /*!40111 SET SQL_NOTESOLD_SQL_NOTES */;