Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

期权链数据缺失 #482

Open
SAMPLE-42 opened this issue May 14, 2024 · 1 comment
Open

期权链数据缺失 #482

SAMPLE-42 opened this issue May 14, 2024 · 1 comment

Comments

@SAMPLE-42
Copy link

DataDownloader下载器指定历史区间,选定’2015-01-01‘ --- 至今,下载所有合约时,下载结果中存在合约缺失(多数为郑商所数据和股指期权数据),请问是什么问题。总共缺失3000+,其中存在一些较近的时间的期权也缺失。使用DataDownloader下载器单独下载缺失合约数据也是空的,例如CZCE.SM403C5300

@SAMPLE-42
Copy link
Author

def fetch_data(self, contract_symbols, start_date, end_date, duration, update_date=False):
    """
        功能:获取指定合约,指定时间区间的历史数据
        Args:
            contract_symbols: 合约代码, list
            start_date: 下载的起始时间
            end_date: 下载的结束时间
            duration: k线级别
            update_date: 数据更新, bool
        return: 历史行情数据, dataframe
    """

    data_dir_path = os.path.join(self.root_dir, 'data', f'update_{update_date}')

    if not os.path.exists(data_dir_path):
        os.makedirs(data_dir_path)

    download_data_tasks = {}

    for symbol in tqdm(contract_symbols):

        print(f'当前下载{symbol}合约数据')

        data_save_file_name = f"{symbol}_data.csv"
        data_save_file_path = os.path.join(data_dir_path, data_save_file_name)

        download_data_tasks[symbol] = DataDownloader(api=self.api,
                                                     symbol_list=symbol,
                                                     dur_sec=duration,
                                                     start_dt=start_date,
                                                     end_dt=end_date,
                                                     csv_file_name=data_save_file_path,
                                                     )

    with closing(self.api):
        while not all([v.is_finished() for v in download_data_tasks.values()]):
            self.api.wait_update()
        print("progress: ", {k: ("%.2f%%" % v.get_progress()) for k, v in download_data_tasks.items()})

        self.logger.log(logging.INFO, f'{start_date}-----{end_date}数据全部下载完成')
        print(f'{start_date}-----{end_date}数据全部下载完成')

    time.sleep(10)
    # 检查是否有空的DataDownloader,如果有则删除相应文件

    for symbol, downloader in download_data_tasks.items():
        # 构建文件路径
        file_csv_path = os.path.join(data_dir_path, f"{symbol}_data.csv")

        if os.path.exists(file_csv_path):
            df = pd.read_csv(file_csv_path)

            if not df.empty:
                df = pd.read_csv(file_csv_path)
                df_copy = df.copy()
                # df_copy.columns = ['datetime', 'open', 'high', 'low', 'close', 'volume', 'open_oi', 'close_oi']
                df_copy['instrument_id'] = symbol
                df_copy.to_csv(file_csv_path)

            else:
                os.remove(file_csv_path)
                self.logger.log(logging.INFO, f"删除空文件: {symbol}")
                print(f"删除空文件: {symbol}")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant