引言1.申请KEY2.文档读取3.接口提交4.提取结果5.写入文档参考地址:完整代码:
引言
最近爬虫了一堆新冠病的数据,想要通过FineBI在地图上展示出来,但是需要自行提供经纬度,网上倒是有部分现成的,比如《Python版中国省市经纬度》,需要可自取。当然还是建议自己爬取,一通百通。
效果如下:
代码功能:
json文本转换与关键字段提取csv文件创建和数值写入list值批量读取和切割转换
1.申请KEY
高德上已有相应的开发手册介绍,直接参考其进行相应操作即可。
地理编码/逆地理编码 API:
/api/webservice/guide/api/georegeo
2.文档读取
之前爬的地址数据我放在a1.csv中,字段名称是’省份’和’城市’,文档编码是’gb2312
’
def parse():datas = []m = 0lists=[]totalListData = pd.read_csv('locs.csv', encoding='gb2312')totalListDict = totalListData.to_dict('index')for j in range(0, len(totalListDict)):datas.append(#加'|'是为了后面方便批量查询str(totalListDict[j]['省份']+'省'+str(totalListDict[j]['城市'])+'|'))#将数组切割为每十个一组lists = [datas[i:i + 10] for i in range(0, len(datas), 10)]return lists
3.接口提交
将获取到的地址名称通过高德api实现转换;
需设置batch
值为true
,方便批量查询
def transform(cityName):ak = 'your_key_value'#此处填写自己申请的key值base = "/v3/geocode/geo?key=%s&address=%s&batch=true" % (ak, cityName)
4.提取结果
仅仅获取结果还不够,要将我们需要的经纬度信息提取出来。
根据接口文档可知,经纬度信息在geocodes
字段中的location
中;
def transform(k,cityName):ak = 'yourkey' #被人直接复制粘贴了,搞得我自己没了次数base = "/v3/geocode/geo?key=%s&address=%s&batch=true" % (ak, cityName)response = requests.get(base)answer = response. json()j=0list=[]while (j< int(answer['count'])):# 不过爬取过程中出现有的城市无法确认经纬度的问题,加入判定语句,直接剔除掉if ((answer['geocodes'] != []) and (answer['geocodes'][j]['district'] != [])):#省province=answer['geocodes'][j]['province']#市city=answer['geocodes'][j]['city']location=answer['geocodes'][j]['location']location=''.join(location).split(',',1)#经度jingdu=location[0]#纬度weidu=location[1]#地区district=answer['geocodes'][j]['district']df.loc[k] = [province,city,district,jingdu,weidu]#换行k=k+1j=j+1
5.写入文档
如下,写入localdetail.csv:
if __name__ == '__main__':df = pd.DataFrame(columns=['province','city', 'district','jingdu','weidu'])cityNames = parse()k=0for cityName in cityNames:transform(k,cityName)k=k+10df.to_csv('locdetail7.csv', index=False)
最后贴图尚未完成FineBI的展示效果:
参考地址:
《Python 利用高德地图api实现经纬度与地址的批量转换》
《利用高德地图api将实际地址转换为经纬度(高德坐标系)python》
完整代码:
# -*- coding:utf-8 -*-'''利用高德地图api实现经纬度与地址的批量转换'''import requestsimport pandas as pdimport timeimport importlibimport sysimport jsonimportlib.reload(sys)def parse():datas = []m = 0lists=[]totalListData = pd.read_csv('Updates_NC.csv', encoding='gb2312')totalListDict = totalListData.to_dict('index')for j in range(0, len(totalListDict)):datas.append(str(totalListDict[j]['省份']+'省'+str(totalListDict[j]['城市'])+'|'))lists = [datas[i:i + 10] for i in range(0, len(datas), 10)]return listsdef transform(k,cityName):ak = 'your_key_value'base = "/v3/geocode/geo?key=%s&address=%s&batch=true" % (ak, cityName)response = requests.get(base)answer = response. json()j=0list=[]while (j< int(answer['count'])):if ((answer['geocodes'] != []) and (answer['geocodes'][j]['district'] != [])):province=answer['geocodes'][j]['province']city=answer['geocodes'][j]['city']location=answer['geocodes'][j]['location']location=''.join(location).split(',',1)jingdu=location[0]weidu=location[1]district=answer['geocodes'][j]['district']df.loc[k] = [province,city,district,jingdu,weidu]k=k+1j=j+1if __name__ == '__main__':df = pd.DataFrame(columns=['province','city', 'district','jingdu','weidu'])cityNames = parse()k=0for cityName in cityNames:transform(k,cityName)k=k+10df.to_csv('locdetail7.csv', index=False)
github地址:
/wangjie182/wh_ncov_city