Coding Period: Week 6
Wrote readme of scraper, Scraped CDM of California
Wrote Readme.md of scraper
I worked on readme.md of scraper, I wrote in detail about project & then how to setup project in your system. I have written the commands which will be needed for scraping data. Then user needs to process the scraped CDM.
Till now I have scraped & processed data of three states Alaska, Indiana, New York.
Scraped CDM of California
I made scrapy spider which gets data from this (Link) , This data is provided by California State Health Department. I have to make "process.py" scripts for each hospital.
Changed Code Organisation
Separated “process.py” from “Data” Directory, so that user don’t need to clone large repository. I have made changes in all process.py scripts.
Working on changes suggested by Mentor Mua
I am currently working on changes suggested by Mentor Mua on Cost Of Care App, I will make Merge Request by tomorrow
Setting Gitlab CI
I have tried a lot but chromedriver & chromebrowser are not working in Linux CI Pipeline, I am getting exception in pipeline,but its working on my windows, maybe I am not installing them properly , I will search more on this, & will discuss with my mentors
What do I plan to do next week?
For Web Scraper
- Generating “process.py” script for each hospital
For App
- Working on View CDM Statewise screen.
GitLab CI Pipeline
- I have to setup chromedriver & chromebrowser in CI Pipeline & then make this project run properly