Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CAL scraper failed in new website #793

Open
niv280 opened this issue Jun 30, 2023 · 7 comments
Open

CAL scraper failed in new website #793

niv280 opened this issue Jun 30, 2023 · 7 comments

Comments

@niv280
Copy link

niv280 commented Jun 30, 2023

Hi,
I'm getting Timeout error while using visacal scraper.
It's look that the error happened inside getLoginFrame, when the scraper don't found the login frame.

I'm not got good knowledge in scraping but it's seems to me that visa cal update their website, and when I tried to use the scraper with showBrowser it's look that there is some kind of popup window that might block the scraper from moving on to the transactions area.

@baruchiro
Copy link
Collaborator

VisaCal is working well for me

@niv280
Copy link
Author

niv280 commented Jul 2, 2023

That can be different between different users?

@galbarm
Copy link

galbarm commented Jul 2, 2023

Maybe different OS? for me this is working well on Windows

@niv280
Copy link
Author

niv280 commented Jul 2, 2023

I'm running it in ubuntu, do you think that can cause problems?
I remember now, that visa cal worked for me in windows when I used Caspion a while ago

@JuloWaks
Copy link

I also have this issue, it seems to be casued by this banner:
Screenshot 2023-07-10 at 16 59 13
I tried accepting the banner on a different browser, but it sill appears.
Scrapper fails with error:

scraping Visa Cal
Visa Cal: START_SCRAPING
Visa Cal: INITIALIZING
Visa Cal: LOGGING_IN
Visa Cal: TERMINATING
Visa Cal: END_SCRAPING
success: false
error type: TIMEOUT
error: wait for iframe with login form
Error: wait for iframe with login form
    at scrape (/Users/jwaksberg/YNAB/israeli-ynab-updater/src/scrape/scrape-base.js:67:11)

@baruchiro
Copy link
Collaborator

I tried to close this banner here:

try {
await waitUntilElementFound(this.page, 'button.btn-close');
const currentUrl = await getCurrentUrl(this.page);
if (currentUrl.endsWith('site-tutorial')) {
await clickButton(this.page, 'button.btn-close');
}
} catch (e) {
const currentUrl = await getCurrentUrl(this.page);
if (currentUrl.endsWith('dashboard')) return;
throw e;
}

@gczobel
Copy link
Contributor

gczobel commented Aug 12, 2023

The same happens to me today, running on ubuntu.
I entered the website in my browser and a change password was required (that was not detected by the scraper). After that, the same screen about the new website was displayed... so probably a combination of the two?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants