AI专业者开发社区 1 号成员
  •  0 回帖  •  9 浏览  •  5 个月前
 

python模拟登陆之下载

好长时间没有更新博客了，哈哈。

今天公司给了这么一个需求，现在我们需要去淘宝获取上一天的订单号，然后再根据订单号去另一个接口去获取订单详情，然后再给我展示到 web！

中间涉及到的技术点有：

  • 模拟登陆
  • 模拟下载
  • 解析 exal 文件数据流
  • 读取 exal 文件，拿出订单号
  • 还有最后一点请求接口

下面就给大家挨个说一下, 刚拿到需求其实还是很模糊的，因为一个都没做过，等静下心来去理解的时候，发现并没有那么难，反而很简单

模拟登陆

一、分析页面请求头

本次登陆地址是 https://huoche.alitrip.com/hello.htm

1、先登陆了一遍查看了一下请求头，发现就携带了三个东西，隐藏 token，用户名，密码

 

一看一目了然，就一个后台页面，可想而知相对来说还是很简单，哈哈，下一步我只需要封装一下 cookie，然后带上 tocken，username，passwd 去登陆咯

给大家说下，python 的 requests 模块可以忽略 cookie，自己创建一个 session 对象，他自己去给咱们匹配 cookie，不用去挨个试 cookie，这样就节省了好多代码和时间

2、代码如下

class TbTomas(object):
    def __init__(self):
        # 配置初始化
        self.session_obj = requests.session()

</span><span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> download_file(self,thomas_username,thomas_password,):
    hello_url </span>= <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/hello.htm</span><span style="color: rgba(128, 0, 0, 1)">'</span>
    <span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 获取原文</span>
    hello_response =<span style="color: rgba(0, 0, 0, 1)"> self.session_obj.get(hello_url)
    </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 正则匹配原文</span>
    h_u_s = re_search(<span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">&lt;input type="hidden" id="h_u_s" name="h_u_s" value="(.*?)"&gt;</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">, hello_response.text)
    
    h_u_s </span>=<span style="color: rgba(0, 0, 0, 1)"> base64.b64encode(h_u_s)
    headers </span>=<span style="color: rgba(0, 0, 0, 1)"> {
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Accept</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">text/html, application/xhtml+xml, image/jxr, */*</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Referer</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/hello.htm</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Accept-Language</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">zh-CN</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,

        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">User-Agent</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Content-Type</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">application/x-www-form-urlencoded</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,

        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Accept-Encoding</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">gzip, deflate</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Host</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">huoche.alitrip.com</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Content-Length</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">73</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Connection</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Keep-Alive</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Cache-Control</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">no-cache</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">
    }

    post_data </span>=<span style="color: rgba(0, 0, 0, 1)"> {
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">h_u_s</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: base64.b64encode(h_u_s),
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">h_u_n</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: thomas_username,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">h_u_p</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: base64.b64encode(thomas_password)
    }
    index_url </span>= <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/index.htm</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">
    index_response </span>= self.session_obj.post(index_url, headers=headers, data=post_data)</pre>

最后一提交 post 请求，就可以判断有没有登录成功了，是不是很简单，哈哈！

数据下载

下载也是和登录是一样的道理，下载的时候肯定也是像网页发一个 post 请求，然后就回去下载 exal 文件咯，python 有这么一个模块 xlrd，可以去操作 exal 文件，非常方便

1、原文是让我们输入时间看，下载那一天的数据，领导给的任务是下载前一天的，所以上一天时间要写几行代码来实现

代码如下：

today = datetime.datetime.now()
yesterday = today + datetime.timedelta(days=-1)
trade_date = yesterday.strftime('%Y-%m-%d')

2、查看下载文件请求的 url，以及提交的数据，一张图一切都明白了

从图中可以看到，该文发送的 url，请求方式，请求头，和返回的数据

3、模拟请求下载，只需用提交一下日期就 OK 搞定，文件下载完毕，接下开要读文件拿自己想要的东西啦

        post_data = {
            'orderExportDate': trade_date
        }
        sheet_content = ""
        for _ in xrange(3):
            try:
                # 得到 exal 文件流
                download_response = self.session_obj.post(download_url, data=post_data)
                # 打开 exal 文件
                xls_content = xlrd.open_workbook(file_contents=download_response.content)
                sheet_content = xls_content.sheets()[0]
                break
            except Exception as e:
                continue

4、这个就众所周知，和读取文件一样，for 循环一行一行读取，然后把订单号挨个添加给一个列表啥啦乱七八糟的

        order_item = []
        for line_num in range(sheet_content.nrows):
            line_item = sheet_content.row_values(line_num)
            if line_item[2]:
                order_item.append(line_item[2], )  # 订单号 order_no
        # 获取到所有订单号
        order_item = order_item[1:]

拿到订单号要去获取订单详情了，但是领导给我说这个已经有同事写好代码了，只需要调用那个接口就好，所以别人的代码我就不往上面展示了，原理很简单

requests 模块，请求 url，get 传入订单号，发送请求，就可以返回数据咯，web 页面展示，那个需求，每个公司都不一样，存入数据库，自己取自己想要的吧。

本文就到这里吧，学到一点东西的请点赞，哈哈

最后附带源码，用户名和密码就不告诉大家啦，啊哈哈

#!/usr/bin/python
# coding:utf-8
import sys
import os
import django

reload(sys)
sys.setdefaultencoding('utf8')
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(file))))  # 把 manage.py 所在目录添加到系统目录
os.environ['DJANGO_SETTINGS_MODULE'] = 'business.settings'  # 设置 setting 文件
django.setup()  # 初始化 Django 环境


import requests
import re
import logging
import base64
import xlrd
import datetime
import time
import MySQLdb
import threadpool
from business import settings
from train.depends.platform import Platform
from train.models import TbTomasOrder,TbTomasEpay,TtTicketThomas,TbTomasLinkman
from train import utils
from train.status import OrderStatus
from django.core.mail import EmailMultiAlternatives
from train.busi import insert_order,insert_ticket,insert_epay,insert_linkman


logger = logging.getLogger('django')


class TbTomas(object):


succ_number </span>=<span style="color: rgba(0, 0, 0, 1)"> 0
fail_number </span>=<span style="color: rgba(0, 0, 0, 1)"> 0
fail_order </span>=<span style="color: rgba(0, 0, 0, 1)"> []
</span><span style="color: rgba(0, 0, 255, 1)">def</span> <span style="color: rgba(128, 0, 128, 1)">__init__</span>(self,thread_num = 3<span style="color: rgba(0, 0, 0, 1)">):
    </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 配置初始化</span>
    self.session_obj =<span style="color: rgba(0, 0, 0, 1)"> requests.session()
    self.fail_order </span>=<span style="color: rgba(0, 0, 0, 1)"> []
    self.succ_number </span>=<span style="color: rgba(0, 0, 0, 1)"> 0
    self.fail_number </span>=<span style="color: rgba(0, 0, 0, 1)"> 0
    self.thread_num </span>=<span style="color: rgba(0, 0, 0, 1)"> thread_num
    self.start_date </span>= <span style="color: rgba(128, 0, 0, 1)">""</span><span style="color: rgba(0, 0, 0, 1)">
    self.end_date </span>= <span style="color: rgba(128, 0, 0, 1)">""</span><span style="color: rgba(0, 0, 0, 1)">
    self.trade_date </span>=<span style="color: rgba(0, 0, 0, 1)"> utils.now()

</span><span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> login_thomas(self,thomas_username,thomas_password):
    hello_url </span>= <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/hello.htm</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">
    hello_response </span>=<span style="color: rgba(0, 0, 0, 1)"> self.session_obj.get(hello_url)
    h_u_s </span>= re_search(<span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">&lt;input type="hidden" id="h_u_s" name="h_u_s" value="(.*?)"&gt;</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">, hello_response.text)
    h_u_s </span>=<span style="color: rgba(0, 0, 0, 1)"> base64.b64encode(h_u_s)
    headers </span>=<span style="color: rgba(0, 0, 0, 1)"> {
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Accept</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">text/html, application/xhtml+xml, image/jxr, */*</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Referer</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/hello.htm</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Accept-Language</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">zh-CN</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,

        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">User-Agent</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Content-Type</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">application/x-www-form-urlencoded</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,

        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Accept-Encoding</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">gzip, deflate</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Host</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">huoche.alitrip.com</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Content-Length</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">73</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Connection</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Keep-Alive</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">Cache-Control</span><span style="color: rgba(128, 0, 0, 1)">'</span>: <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">no-cache</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">
    }

    post_data </span>=<span style="color: rgba(0, 0, 0, 1)"> {
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">h_u_s</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: base64.b64encode(h_u_s),
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">h_u_n</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: thomas_username,
        </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">h_u_p</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: base64.b64encode(thomas_password)
    }
    index_url </span>= <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/index.htm</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">
    index_response </span>= self.session_obj.post(index_url, headers=headers, data=<span style="color: rgba(0, 0, 0, 1)">post_data)
    logger.info(u</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">登陆成功，等待下载文件...</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)


</span><span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> download_file(self,thomas_username,thomas_password,args):
    </span><span style="color: rgba(0, 0, 255, 1)">for</span> _ <span style="color: rgba(0, 0, 255, 1)">in</span> xrange(3<span style="color: rgba(0, 0, 0, 1)">):
        </span><span style="color: rgba(0, 0, 255, 1)">try</span><span style="color: rgba(0, 0, 0, 1)">:
            self.login_thomas(thomas_username,thomas_password)
            </span><span style="color: rgba(0, 0, 255, 1)">break</span>
        <span style="color: rgba(0, 0, 255, 1)">except</span><span style="color: rgba(0, 0, 0, 1)"> Exception as e:
            logger.error(e)
            </span><span style="color: rgba(0, 0, 255, 1)">continue</span>

    <span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 处理时间</span>
    all_time =<span style="color: rgba(0, 0, 0, 1)"> self.date_time_handle(args)
    </span><span style="color: rgba(0, 0, 255, 1)">if</span> <span style="color: rgba(0, 0, 255, 1)">not</span><span style="color: rgba(0, 0, 0, 1)"> all_time:
        logger.error(u</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">日期格式错误!!</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)
        </span><span style="color: rgba(0, 0, 255, 1)">return</span>
    <span style="color: rgba(0, 0, 255, 1)">for</span> trade_date <span style="color: rgba(0, 0, 255, 1)">in</span><span style="color: rgba(0, 0, 0, 1)"> all_time:
        </span><span style="color: rgba(0, 0, 255, 1)">try</span><span style="color: rgba(0, 0, 0, 1)">:
            self.trade_date </span>=<span style="color: rgba(0, 0, 0, 1)"> trade_date
            post_data </span>=<span style="color: rgba(0, 0, 0, 1)"> {
                </span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">orderExportDate</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">: trade_date
            }
            download_url </span>= <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">https://huoche.alitrip.com/orderlistexp.do</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">
            sheet_content </span>= <span style="color: rgba(128, 0, 0, 1)">""</span>
            <span style="color: rgba(0, 0, 255, 1)">for</span> _ <span style="color: rgba(0, 0, 255, 1)">in</span> xrange(3<span style="color: rgba(0, 0, 0, 1)">):
                </span><span style="color: rgba(0, 0, 255, 1)">try</span><span style="color: rgba(0, 0, 0, 1)">:
                    </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 得到exal文件流</span>
                    download_response = self.session_obj.post(download_url, data=<span style="color: rgba(0, 0, 0, 1)">post_data)
                    </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 打开exal文件</span>
                    xls_content = xlrd.open_workbook(file_contents=<span style="color: rgba(0, 0, 0, 1)">download_response.content)
                    sheet_content </span>=<span style="color: rgba(0, 0, 0, 1)"> xls_content.sheets()[0]
                    logger.info(u</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">下载文件成功，正在拿取订单号</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)
                    </span><span style="color: rgba(0, 0, 255, 1)">break</span>
                <span style="color: rgba(0, 0, 255, 1)">except</span><span style="color: rgba(0, 0, 0, 1)"> Exception as e:
                    logger.error(u</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">下载文件超时，正在等待重新登录后下载...</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)
                    self.login_thomas(thomas_username, thomas_password)
                    </span><span style="color: rgba(0, 0, 255, 1)">continue</span><span style="color: rgba(0, 0, 0, 1)">
            order_item </span>=<span style="color: rgba(0, 0, 0, 1)"> []
            </span><span style="color: rgba(0, 0, 255, 1)">if</span> <span style="color: rgba(0, 0, 255, 1)">not</span><span style="color: rgba(0, 0, 0, 1)">  sheet_content:
                logger.error(u</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">下载文件失败，正在重新登录...</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">)
                </span><span style="color: rgba(0, 0, 255, 1)">continue</span>
            <span style="color: rgba(0, 0, 255, 1)">for</span> line_num <span style="color: rgba(0, 0, 255, 1)">in</span><span style="color: rgba(0, 0, 0, 1)"> range(sheet_content.nrows):
                line_item </span>=<span style="color: rgba(0, 0, 0, 1)"> sheet_content.row_values(line_num)
                </span><span style="color: rgba(0, 0, 255, 1)">if</span> line_item[2] <span style="color: rgba(0, 0, 255, 1)">and</span> line_item[2] <span style="color: rgba(0, 0, 255, 1)">not</span> <span style="color: rgba(0, 0, 255, 1)">in</span><span style="color: rgba(0, 0, 0, 1)"> order_item:
                    order_item.append(line_item[</span>2], )  <span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 订单号 order_no</span>

            <span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 获取到所有订单号</span>
            order_item = order_item[1<span style="color: rgba(0, 0, 0, 1)">:]
            </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 根据订单号去拿订单详情</span>
            logger.info(u<span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">正在写入数据库</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)

            </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 多线程去执行</span>
            pool =<span style="color: rgba(0, 0, 0, 1)"> threadpool.ThreadPool(self.thread_num)
            reqs </span>=<span style="color: rgba(0, 0, 0, 1)"> threadpool.makeRequests(self.create_order_info, order_item)
            [pool.putRequest(req) </span><span style="color: rgba(0, 0, 255, 1)">for</span> req <span style="color: rgba(0, 0, 255, 1)">in</span><span style="color: rgba(0, 0, 0, 1)"> reqs]
            pool.wait()
            logger.info(u</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">写入完成，完成时间为：%s</span><span style="color: rgba(128, 0, 0, 1)">'</span>%<span style="color: rgba(0, 0, 0, 1)"> self.trade_date)

            content </span>=<span style="color: rgba(0, 0, 0, 1)"> self.add_content(len(order_item), self.succ_number, self.fail_number, self.fail_order)
            self.send_mail(content</span>=<span style="color: rgba(0, 0, 0, 1)">content)
            self.succ_number,self.fail_order </span>=<span style="color: rgba(0, 0, 0, 1)"> 0,0
            self.fail_order </span>=<span style="color: rgba(0, 0, 0, 1)"> []
            </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> self.create_order_info(order_item)</span>
        <span style="color: rgba(0, 0, 255, 1)">except</span><span style="color: rgba(0, 0, 0, 1)"> Exception as e:
            logger.error(e)
</span><span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> date_time_handle(self,args):
    all_time </span>=<span style="color: rgba(0, 0, 0, 1)"> []
    </span><span style="color: rgba(0, 0, 255, 1)">if</span><span style="color: rgba(0, 0, 0, 1)"> args:
        </span><span style="color: rgba(0, 0, 255, 1)">if</span> len(args) == 1<span style="color: rgba(0, 0, 0, 1)">:
            self.start_date </span>= datetime.datetime.strptime(args[0], <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">).date()
            self.end_date </span>= datetime.datetime.strptime(datetime.datetime.now().strftime(<span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span>), <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">).date()
        </span><span style="color: rgba(0, 0, 255, 1)">elif</span> len(args) == 2<span style="color: rgba(0, 0, 0, 1)">:
            self.start_date </span>= datetime.datetime.strptime(args[0], <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">).date()
            self.end_date </span>= datetime.datetime.strptime(args[1], <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">).date()
        </span><span style="color: rgba(0, 0, 255, 1)">elif</span> len(args) == 3<span style="color: rgba(0, 0, 0, 1)">:
            self.start_date </span>= datetime.datetime.strptime(args[0], <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">).date()
            self.end_date </span>= datetime.datetime.strptime(args[1], <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">).date()
            self.thread_num </span>= int(args[2<span style="color: rgba(0, 0, 0, 1)">])
        </span><span style="color: rgba(0, 0, 255, 1)">else</span><span style="color: rgba(0, 0, 0, 1)">:
            logger.error(u</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">传入参数错误，请重新执行</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)
            </span><span style="color: rgba(0, 0, 255, 1)">return</span><span style="color: rgba(0, 0, 0, 1)">
        i </span>=<span style="color: rgba(0, 0, 0, 1)"> 0
        </span><span style="color: rgba(0, 0, 255, 1)">while</span><span style="color: rgba(0, 0, 0, 1)"> True:
            tomoary </span>= self.start_date + datetime.timedelta(days=<span style="color: rgba(0, 0, 0, 1)">i)
            trade_date </span>= tomoary.strftime(<span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">)
            all_time.append(trade_date)
            i </span>+= 1
            <span style="color: rgba(0, 0, 255, 1)">if</span> tomoary ==<span style="color: rgba(0, 0, 0, 1)"> self.end_date:
                </span><span style="color: rgba(0, 0, 255, 1)">break</span>
    <span style="color: rgba(0, 0, 255, 1)">else</span><span style="color: rgba(0, 0, 0, 1)">:
        today </span>=<span style="color: rgba(0, 0, 0, 1)"> datetime.datetime.now()
        yesterday </span>= today + datetime.timedelta(days=-1<span style="color: rgba(0, 0, 0, 1)">)
        trade_date </span>= yesterday.strftime(<span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">)
        all_time.append(trade_date)
    </span><span style="color: rgba(0, 0, 255, 1)">return</span><span style="color: rgba(0, 0, 0, 1)"> all_time

</span><span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> create_order_info(self, order):
    platform_obj </span>=<span style="color: rgba(0, 0, 0, 1)"> Platform()

    order_info </span>=<span style="color: rgba(0, 0, 0, 1)"> platform_obj.get_order(order)
    </span><span style="color: rgba(0, 0, 255, 1)">if</span> <span style="color: rgba(0, 0, 255, 1)">not</span><span style="color: rgba(0, 0, 0, 1)"> order_info:
        self.fail_order.append(order)
        self.fail_number </span>+= 1<span style="color: rgba(0, 0, 0, 1)">
        logger.error(</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">获取订单号:[%s]失败</span><span style="color: rgba(128, 0, 0, 1)">'</span>%<span style="color: rgba(0, 0, 0, 1)">order)
        </span><span style="color: rgba(0, 0, 255, 1)">return</span>

    <span style="color: rgba(0, 0, 255, 1)">try</span><span style="color: rgba(0, 0, 0, 1)">:
        </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 插入order表</span>
        <span style="color: rgba(0, 0, 255, 1)">if</span> TbTomasOrder.objects.filter(order_no=<span style="color: rgba(0, 0, 0, 1)">order).exists():
            logger.error(</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">订单号:[%s]已经存在于TbTomasOrder</span><span style="color: rgba(128, 0, 0, 1)">'</span>%<span style="color: rgba(0, 0, 0, 1)">order)
            self.fail_order.append(order)
            self.fail_number </span>+= 1
            <span style="color: rgba(0, 0, 255, 1)">return</span>
        <span style="color: rgba(0, 0, 255, 1)">else</span><span style="color: rgba(0, 0, 0, 1)">:
            insert_order(order_info,order,self.trade_date)
            self.succ_number </span>+= 1

        <span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 插入ticket表</span>



            insert_ticket(order_info,order,self.trade_date)


        </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 插入联系人</span>
        <span style="color: rgba(0, 0, 255, 1)">if</span> TbTomasLinkman.objects.filter(order_no=<span style="color: rgba(0, 0, 0, 1)">order).exists():
            logger.error(</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">订单号:[%s]已经存在于TbTomasLinkman</span><span style="color: rgba(128, 0, 0, 1)">'</span>%<span style="color: rgba(0, 0, 0, 1)">order)
        </span><span style="color: rgba(0, 0, 255, 1)">else</span><span style="color: rgba(0, 0, 0, 1)">:
            insert_linkman(order_info,order,self.trade_date)

        </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 插入epay表</span>
        <span style="color: rgba(0, 0, 255, 1)">if</span> TbTomasEpay.objects.filter(order_no=<span style="color: rgba(0, 0, 0, 1)">order).exists():
            logger.error(</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">订单号:[%s]已经存在于TbTomasEpay</span><span style="color: rgba(128, 0, 0, 1)">'</span>%<span style="color: rgba(0, 0, 0, 1)">order)
        </span><span style="color: rgba(0, 0, 255, 1)">else</span><span style="color: rgba(0, 0, 0, 1)">:
            insert_epay(order_info,order,self.trade_date)

    </span><span style="color: rgba(0, 0, 255, 1)">except</span><span style="color: rgba(0, 0, 0, 1)"> Exception as e:
        logger.error(e)
        self.fail_number </span>+=1



<span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> add_content(self,total,succ_number,fail_number,fail_order):
    content </span>= u<span style="color: rgba(128, 0, 0, 1)">'''</span><span style="color: rgba(128, 0, 0, 1)">
    &lt;h3&gt;托马斯导入订单报表&lt;/h3&gt;
    &lt;div class="col-xs-12"&gt;
        &lt;table border="1" cellpadding="3" cellspacing="1"&gt;
            &lt;tr&gt;
                &lt;td&gt;日期&lt;/td&gt;
                &lt;td&gt;总单数&lt;/td&gt;
                &lt;td&gt;成功单数&lt;/td&gt;
                &lt;td&gt;失败单数&lt;/td&gt;
                &lt;td&gt;失败订单号&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;%s&lt;/td&gt;
                &lt;td&gt;%s&lt;/td&gt;
                &lt;td&gt;%s&lt;/td&gt;
                &lt;td&gt;%s&lt;/td&gt;
                &lt;td&gt;%s&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/table&gt;
    &lt;/div&gt;
    </span><span style="color: rgba(128, 0, 0, 1)">'''</span>%(datetime.datetime.now().strftime(<span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">%Y-%m-%d %H:%M</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">),total,succ_number,fail_number,fail_order)
    </span><span style="color: rgba(0, 0, 255, 1)">return</span><span style="color: rgba(0, 0, 0, 1)"> content

</span><span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> send_mail(self, content):
    time_target </span>=<span style="color: rgba(0, 0, 0, 1)"> self.trade_date
    subject </span>= u<span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">托马斯数据抓取邮件 %s</span><span style="color: rgba(128, 0, 0, 1)">'</span> %<span style="color: rgba(0, 0, 0, 1)"> (time_target)

    logger.info(u</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">准备发送邮件....%s</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">, subject)
    mail_address </span>=<span style="color: rgba(0, 0, 0, 1)"> settings.mail_address_thomas

    to_addr </span>=<span style="color: rgba(0, 0, 0, 1)"> []

    </span><span style="color: rgba(0, 0, 255, 1)">if</span><span style="color: rgba(0, 0, 0, 1)"> isinstance(mail_address, list):
        to_addr </span>+=<span style="color: rgba(0, 0, 0, 1)"> mail_address
    </span><span style="color: rgba(0, 0, 255, 1)">elif</span><span style="color: rgba(0, 0, 0, 1)"> isinstance(mail_address, str):
        to_addr.append(mail_address)

    logger.debug(to_addr)

    from_email </span>=<span style="color: rgba(0, 0, 0, 1)"> settings.DEFAULT_FROM_EMAIL
    msg </span>= EmailMultiAlternatives(subject, <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">result</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">, from_email, to_addr)
    msg.attach_alternative(content, </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">text/html</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">)
    flag </span>=<span style="color: rgba(0, 0, 0, 1)"> msg.send()
    </span><span style="color: rgba(0, 0, 255, 1)">if</span><span style="color: rgba(0, 0, 0, 1)"> flag:
        logger.info(u</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">%s发送成功</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">, subject)
    </span><span style="color: rgba(0, 0, 255, 1)">else</span><span style="color: rgba(0, 0, 0, 1)">:
        logger.error(u</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">%s发送失败</span><span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(0, 0, 0, 1)">, subject)
    </span><span style="color: rgba(0, 0, 255, 1)">return</span>

<span style="color: rgba(0, 0, 255, 1)">def</span><span style="color: rgba(0, 0, 0, 1)"> run(self, username,passwd,args):
    </span><span style="color: rgba(0, 128, 0, 1)">#</span><span style="color: rgba(0, 128, 0, 1)"> 登陆托马斯后台</span>
    <span style="color: rgba(0, 0, 255, 1)">for</span> _ <span style="color: rgba(0, 0, 255, 1)">in</span> xrange(3<span style="color: rgba(0, 0, 0, 1)">):
        </span><span style="color: rgba(0, 0, 255, 1)">try</span><span style="color: rgba(0, 0, 0, 1)">:
            self.download_file(username,passwd,args)
            </span><span style="color: rgba(0, 0, 255, 1)">break</span>
        <span style="color: rgba(0, 0, 255, 1)">except</span><span style="color: rgba(0, 0, 0, 1)"> Exception as e:
            logger.error(e)
            </span><span style="color: rgba(0, 0, 255, 1)">continue</span>



def re_search(regex, subject):
subject = str(subject)
obj = re.compile(regex)
match = obj.search(subject)
if match:
result = match.group(1)
else:
result = ''
return result


def main():
username = base64.b64decode(settings.THOMAS_USERNAME)
passwd = base64.b64decode(settings.THOMAS_PASSWORD)
args = sys.argv[1:] if sys.argv[1:] else ""
TbTomas().run(username,passwd,args)


if name == "main":
main()

thread_code