This has turned into an insane situation. The forum won’t let me post any pictures of our conversation, which is upsetting, but this has gone haywire.
About a month and a half ago, my dog died. I posted on redditgetsdrawnbadly for some silly drawings. I haven’t really posted much, but I’ve seen people do that and it seems to make them feel better. Her death was so sudden, I miss her so much. Anyways, this user commented on the post about how she did a watercolor painting and would do another to make me smile. I was overjoyed and offered another photo to make her painting easier. She messaged me a picture of the paintings and I was so happy! She looked like her old chunky self. The user then told me she’d send them to me if I wanted them, for free or “pay/tip what you can, when you receive them”. I gave her an address she could send them to. She said she’d send them off later that day and send me a tracking number. My last message was asking for her Venmo or cashapp since I didn’t mind compensating her well ($150. I am off well enough and wanted to pay her back for her kindness). No response. For one full month, I texted her every week or so asking for an update on the URL. Just a “hey, any update?” Or “Hello?”. No response. I thought I got scammed, and was worried about my safety (gave someone my address, which is my own fault) so I checked her account to see if she had been active. She was very active! That made me upset, so when I saw she commented on someone’s post about “making pet portraits and sending them off as a ‘pay what you want’ “ I fumed. I responded to her comment and basically said “This isn't true. They'll do it but never send it to you and will ghost you when you ask for any updates. They drew my dog who passed away and ghosted me after I gave them my address. Don't waste your time like I did.”
That must have set her off. She messaged me back immediately saying her cat died and she couldn’t send my package because of it. I told her I didn’t need the package sent right away, I just wanted communication. Even a simple, “hey, I’m busy rn I’ll get to you when I can” would’ve soothed my anxiety. Apparently, I’m a terrible person. I was calm and collected in my response to her, but she twisted it around and said “Sorry my cat suddenly you have ruined my Mother's Day I'm bawling. I only tried to do something nice and I got slandered all over. I hope you get it. If I die suddenly soon and you hear about it on Reddit, I hope you know you contributed to it.” (Exact copy and paste)
I suggested seeking professional help. I couldn’t block her because I unblocked her yesterday to respond to her comment. Now I’m waiting for 9pm tonight to block her again.
She has sent people to harass me and call me a scammer. Saying I’m “scamming for free paintings”. All I did was ask for silly doodles!! She then said “she never even paid postage”in a comment about me, which is ironic considering she told me in DMs that I didn’t need to pay a dime (like I said above, I was going to compensate her when I received the paintings, but we never made it that far. I never even got her info). She messaged me saying she’ll escalate this situation if I don’t apologize for slandering her and to never “cross a witch”. Then she said I’ve ruined her health and she’s dealing with cancer and I’ve ruined her spark for drawing because of how much I “slandered” her with my “lies”.
I just want to know where I went wrong here. I wish she’d block me so this can be over. She is blowing up my phone with her friends harassing me, all because I responded to her public comment. Her last text to me was “Guess u decided to FAAFO. Good luck!” I want this over. She’s going to paint me in a terrible light and I’m exhausted from arguing with people online.
So Reddit, please tell me- AITA?

I only rent my lake cabin for 2 months every summer. Because of this, I set my 'Availability window' setting to 'Dates unavailable by default' and manually unblock dates in June and July. When I access my listing from its unique URL, it appears the listing can be booked. However, the listing does not appear in any search results.
After contacting support, they were adamant that I need to change my 'Availability window' to at least '3 months in advance' in order for my listing to appear in search results. I argued this is a bug, why would it be available to book, and not appear in search results?
Has anyone else encountered this? Feels like something broke in airbnb backend recently

Hi all, I was trying to apply cosmetic/CSS filters to specific URLs rather than a whole domain, but I'm not entirely sure if this is possible. Basically, I wanted to block a CSS class on most pages because it was used on an annoying div, but if I went to a specific subpage, I wanted that CSS class to be unblocked so I could see the div. This threw the error in the title.
Whenever I had ## in the filter already, and tried to modify it by adding a slash after the TLD, or add to the start of a URL, it would throw the "Invalid Filter: Bad Domain Error". For example, hello.com/##.class or hello.com###id would return this error.
I was eventually advised that "these filters are applied to all pages on the specified domain" and that "Cosmetic filtering in uBlock Origin primarily works with domain names rather than URL patterns." Can anyone confirm if that's an accurate understanding?
If this is true, then I would like to suggest that the error acknowledges this is in some way because of the cosmetic filtering restricting the input to only domain names, rather than trying to assume that whatever the input given is supposed to be a domain name.
Is this something I should submit to the Github issue tracker? I've never used it before so is it as simple as just filling out the form? Thanks in advance!

(This is a throwaway)
It's fast!

 Speedtest by Ookla Server: TELUS - Vancouver, BC (id: 3049) ISP: Shaw Communications Idle Latency: 2.50 ms (jitter: 0.14ms, low: 2.48ms, high: 3.00ms) Download: 3094.32 Mbps (data used: 3.1 GB) 11.54 ms (jitter: 0.40ms, low: 2.46ms, high: 12.58ms) Upload: 3067.65 Mbps (data used: 1.5 GB) 7.62 ms (jitter: 0.62ms, low: 2.41ms, high: 9.48ms) Packet Loss: Not available. Result URL: https://www.speedtest.net/result/c/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx 

I have had Shaw (now Rogers) 1.5Gb down/150Mb up for many years. Telus sent me a SPRINGSAVINGS promo code for Residential service. The promo was:

1.0Gb up/1.0Gb down for $65/month on a 2-year contract 1.5Gb up/1.5Gb down for $75/month on a 2-year contract 3.0Gb up/3.0Gb down for $85/month on a 2-year contract 

I signed up for 3.0Gb and after a lengthy installation process, I was testing. I never canceled Rogers/Shaw yet. However, the brass tacks of it all is I hate to report that Telus filters incoming ports for Residential service. If this matters to you, it's a big problem. Their security measures policy mentions certain ports are blocked but in fact, all incoming ports seem to be blocked.
From a consumer-like-me point of view, the security policy is stupidly restricting "to protect me". However, if you pay (a lot) more, you can get a business account that does not block any ports. Go figure.
I had 15 days to trial the service and decided to cancel after 5 days. There is no option (I've talked to many Telus departments - tech support tiers, loyalty, business sales, etc) to unblock ports except to go to a business account which doesn't block ports. The business account is over $200/month on a 3-year contract (without a promo). With a promo, I was offered as low as $115/month on a 5-year contract, $135/month on a 3-year contract.
There are a lot of technical details behind my testing but not worth mentioning here. Things may be different on the lower tier speeds but for 3Gb, you receive the Network Access Hub (Technicolor NH20T), an SPF+ XGS PON and the WiFi 6E. The NH20T is what's registered on the Telus back end, so you cannot bypass the NAH, plugging the SPF+ module directly into another device.
It was a fun 5 days... but it wasn't something that was going to work for me in the long run. Again, if having incoming ports open is important to you, don't bother. Rogers/Shaw doesn't block ports.

Hey All,
We have a client who runs a large and popular financial news site. They recently rebranded to a new .com url and we have just found out they seem to be blocked in santander bank.
Looking at their screenshot the block reason is "SCIB - Blocked Categories".
My belief is that it is possible that since the site is new it has not been assigned a category by the firewall vendor and thus is in a "uncategorized" category, which may be blocked by their corporate IT policy. The best course of action is to work with their corporate IT team to get the site unblocked and categorized properly so whatever vendor they are using can propagate it to any other customers who may also use a similar ruleset.
Of course our client and this end user in the bank just want to throw their hands up and not realize why it can't be fixed. They're constantly pushing us for an update and the fact that there must be something we can do.
The only thing I can think of is the fact that these firewalls use url databases. For example, PAN-DB for palo alto firewalls. I am thinking if we can find out what firewall manufacturer they use, we can find out the url list and then submit a category or appeal or something to see if our site is listed appropriately or not at all as a legitimate financial news organization.
I am not sure if they use palo alto or what - and I couldnt see a way to query this "PAN-DB", so I'm here asking for any ideas.
Thanks!

its the end of the school year and we are allowed to play games on the computers for classes we already took the test for. i know its possible to play fortnight somehow and animal jam, and minecraft to because of x box that they haven't blocked anyways how *do* you unblock websites i tried everything: copy and translating url, they blocked all of the vpn and proxys and i watched videos they either dont work or they block videos (that i think worked) any other tips? is it even possible at this point!?

when i put in the url for old character ai, it says i have to unblock challenges.cloudflare.com (on both mobile and desktop! come on i don't like the new site *sob*)
the new site doesn't let me log in properly either :( on mobile it's stuck on the verification page. really sucks. i had all these plot ideas.

We are in the 1% use case where blocking the crawl is more important than deindexing.
I work with a large website that has over 1.4M of query parameter URL that has been blocked by robots.txt. I understand that means there is still a potential of it being indexed - and it is because it is internally linked on our webpage due to other reasons.
What is the solution here to get the page deindexed but also prevent crawling?

1. Unblock from robots, add a noindex tag, wait till they deindex then block them again?
2. Fix the internal linking issue and then block them so google doesn't reach the page?
3. Could adding a nofollow tag to the query parameter help?
4. Some other solution I can't think of and hoping you folks can help

Hi there! So, the IT department where I work implemented a tool called Global Protect which blocks lots of domains in the Games category. So I need to create a ticket for them, asking to unblock some domains for me so I can use all epic games services and Quixel services. So, I wanted to ask you guys, if someone know and could send me all URLs and domains to access all Epic Games services and also Quixel services, since even when we were able to use the EG launcher and UE services, we couldnt use the built in Bridge plugin.
Thank you very much in advance!

Last Edit: April 28, 2024 10:58am EST
Preface: I imagine you've seen this comment already, but I decided to make a post and not need to constantly update the comments for whatever reason. Links below lead to Youtube, Imgur, and Twitter. If you can't access Twitter, I have screenshots available. Hopefully this is all the info, but I imagine there will be more.
I may use the full name or shortened name of the vtubers interchangeably.

• Dovealove = Dove
• RiiKami = Kami
• AndySSJ5 = Andy
• Thintrovert = Thin

Videos and Screenshots

• Kenji's Twitch VOD (3hrs 28min): Addressing everything
• Kenji's Youtube VOD (different VOD/content) (3hrs 18min): Kenji Gets DOXXED & Kage Gets Exposed! LEAKED DM's
• Kenji's Youtube Clip (2:51min) This Vtuber DOXXED Kenji...
• RiiKami video (3:45min): NO LONGER ASSOCIATED.
• Kage video (19:50) : I'm sorry, Goodbye.
• Screenshots of Dovealove, Thintrovert, and Mofongo Boys' Tweets to Kenji
• Screenshots of @/utbahb_ or utbah's Dovealove google doc
• Screenshots of Kenji's Tweets & Reposts About Dovealove & Mofongo Boys Screenshots and Kenji & Sayu's Reconciliation
• Screenshots (Reddit post) of Dove and Kage's Youtube channel changes
• Erundel covers Kenji stream clip & tweets and Dove's response and Mofongo Boys' responses and Mofongo and Kenji Timeline
• StaticTokyo covers Timeline (dox, apology, groupchat, others' stories) and Dove doc, Who knew, and Kenji stream clips with Phoenixx & Kage

Rough Timeline

1. Kenji was doxxed by a secret dox chat with Dove and others
   1. There may be a separate secret hate chat that had Mofongo Boys' Kage, Dovealove, Thin, and others
   2. The "hate group" and "dox group" are claimed to be separate by Thin and Maki uses "Hate group" in their tweet, but not "dox"
2. Kenji revealed his face to "out-dox" the dox chat on Feb 11, 2023 stream(according to Erundel and Kenji tries to play it off)
3. Kenji had a [clip] of him calling Zaion LanZa (now Sayu) a p*dophile and his chat supposedly doxxed [sensitive information]
   1. The [italics] indicates removed links or text. The [clip] or other videos related to Zaion's dox can be found, but doxxing is not allowed on this subreddit and would go against this entire post's purpose.
   2. Due to Kenji's chat supposedly doxxing information on Zaion, some people don't have sympathy for Kenji's own dox.
   3. I cannot confirm which order point 2 and point 3 occur, but Kenji's face reveal was Feb 11 and Zaion's termination tweet was March 10. So the clip likely happened before or around March 10, 2023.
4. Kenji being doxxed is now revealed and Mofongo Boys' members are apologizing over Twitter (and Kage has a stream clip on Erundel's coverage) and cutting ties with Dovealove.
5. Dovealove and Kage seem to be having the worst experience.
   1. Dove confirming themselves of being the doxxer and tweeted by Kenji. Dove is also accused by @/hayayena of spreading rumors and being manipulative.
   2. Kage getting an angry tweet from Kenji and @/OliDeliber accusing him of being "the most awful client"
6. Twitter is going mob mentality on anyone Kenji is against, whether they are confirmed (Dove, Thin, Kage) or simply guilty by association (Phoenixx, AndySSJ5, and RiiKami) or Sayu.
7. Sayu and Kenji have come to an agreement and ask their fans to not harass
   1. Prior to the reconiclliation, there were angry tweets screenshotted and shared on kurosanji (they side with Sayu and don't brigade, just view screenshots) (Kenji tweeting Sayu blocked him, Sayu unblocking to talk, Kenji's tweet response to DM, Sayu's response)
   2. April 23, 2024 EST: Kenji told fans on a Twitch stream to not harass Sayu (Reddit Clip Post)
   3. April 24, 2024 EST: Kenji has fans or trolls using the drama to continue to harass Sayu. Sayu privated her Twitter (Reddit Screenshot Post).
8. Kage posted an apology video
   1. Explains his background experience
   2. He is graduating
   3. Some tried doxxing him
   4. Video description claims that the "hate" and "dox" group chat are separate

Dove, Thin, and Mofongo Boys
The Mofongo Boys were a group of male vtuber friends (RiiKami, Kage, Phoenixx, and AndySSJ5) and other friends (like Dove and Thin) would collab too. RiiKami and Andy also removed all Mofongo Boys channel links on their youtube channel (personally noted on April 18th EST).

• Dovealove laughed at "being cancelled", now has an empty Youtube channel (graduated)
• Thin tweeted a google doc confirming that he was the owner of the hate chat, but the origin was not meant to be a hate chat
• Phoenixx tweets of knowing of Dove doxxing and claims he held Dove responsible, and didn't know of a hate group chat. Kenji says Phoenixx was the one to tell him that Dove wasn't sorry/downplayed the dox. Later on in a VOD, Kenji seems to be on bad terms with Phoenixx
• RiiKami tweets a google doc saying he "wasn't in a group"(supposedly either group) during "when this big thing happened"
• AndySSJ apologized and I personally don't know anything about his involvement, besides being a Mofongo Boy
• Kage was in the hate groupchat with Dove, Thin, and others. He and Kenji are on bad terms.

Last Edit: April 28, 2024 10:58am EST
Preface: I imagine you've seen this comment already, but I decided to make a post and not need to constantly update the comments for whatever reason. Links below lead to Youtube, Imgur, and Twitter. If you can't access Twitter, I have screenshots available. Hopefully this is all the info, but I imagine there will be more.
I may use the full name or shortened name of the vtubers interchangeably.

• Dovealove = Dove
• RiiKami = Kami
• AndySSJ5 = Andy
• Thintrovert = Thin

Videos and Screenshots

• Kenji's Twitch VOD (3hrs 28min): Addressing everything
• Kenji's Youtube VOD (different VOD/content) (3hrs 18min): Kenji Gets DOXXED & Kage Gets Exposed! LEAKED DM's
• Kenji's Youtube Clip (2:51min) This Vtuber DOXXED Kenji...
• RiiKami video (3:45min): NO LONGER ASSOCIATED.
• Kage video (19:50) : I'm sorry, Goodbye.
• Screenshots of Dovealove, Thintrovert, and Mofongo Boys' Tweets to Kenji
• Screenshots of @/utbahb_ or utbah's Dovealove google doc
• Screenshots of Kenji's Tweets & Reposts About Dovealove & Mofongo Boys Screenshots and Kenji & Sayu's Reconciliation
• Screenshots (Reddit post) of Dove and Kage's Youtube channel changes
• Erundel covers Kenji stream clip & tweets and Dove's response and Mofongo Boys' responses and Mofongo and Kenji Timeline
• StaticTokyo covers Timeline (dox, apology, groupchat, others' stories) and Dove doc, Who knew, and Kenji stream clips with Phoenixx & Kage

Rough Timeline

1. Kenji was doxxed by a secret dox chat with Dove and others
   1. There may be a separate secret hate chat that had Mofongo Boys' Kage, Dovealove, Thin, and others
   2. The "hate group" and "dox group" are claimed to be separate by Thin and Maki uses "Hate group" in their tweet, but not "dox"
2. Kenji revealed his face to "out-dox" the dox chat on Feb 11, 2023 stream(according to Erundel and Kenji tries to play it off)
3. Kenji had a [clip] of him calling Zaion LanZa (now Sayu) a p*dophile and his chat supposedly doxxed [sensitive information]
   1. The [italics] indicates removed links or text. The [clip] or other videos related to Zaion's dox can be found, but doxxing is not allowed on this subreddit and would go against this entire post's purpose.
   2. Due to Kenji's chat supposedly doxxing information on Zaion, some people don't have sympathy for Kenji's own dox.
   3. I cannot confirm which order point 2 and point 3 occur, but Kenji's face reveal was Feb 11 and Zaion's termination tweet was March 10. So the clip likely happened before or around March 10, 2023.
4. Kenji being doxxed is now revealed and Mofongo Boys' members are apologizing over Twitter (and Kage has a stream clip on Erundel's coverage) and cutting ties with Dovealove.
5. Dovealove and Kage seem to be having the worst experience.
   1. Dove confirming themselves of being the doxxer and tweeted by Kenji. Dove is also accused by @/hayayena of spreading rumors and being manipulative.
   2. Kage getting an angry tweet from Kenji and @/OliDeliber accusing him of being "the most awful client"
6. Twitter is going mob mentality on anyone Kenji is against, whether they are confirmed (Dove, Thin, Kage) or simply guilty by association (Phoenixx, AndySSJ5, and RiiKami) or Sayu.
7. Sayu and Kenji have come to an agreement and ask their fans to not harass
   1. Prior to the reconiclliation, there were angry tweets screenshotted and shared on kurosanji (they side with Sayu and don't brigade, just view screenshots) (Kenji tweeting Sayu blocked him, Sayu unblocking to talk, Kenji's tweet response to DM, Sayu's response)
   2. April 23, 2024 EST: Kenji told fans on a Twitch stream to not harass Sayu (Reddit Clip Post)
   3. April 24, 2024 EST: Kenji has fans or trolls using the drama to continue to harass Sayu. Sayu privated her Twitter (Reddit Screenshot Post).
8. Kage posted an apology video
   1. Explains his background experience
   2. He is graduating
   3. Some tried doxxing him
   4. Video description claims that the "hate" and "dox" group chat are separate

Dove, Thin, and Mofongo Boys
The Mofongo Boys were a group of male vtuber friends (RiiKami, Kage, Phoenixx, and AndySSJ5) and other friends (like Dove and Thin) would collab too. RiiKami and Andy also removed all Mofongo Boys channel links on their youtube channel (personally noted on April 18th EST).

• Dovealove laughed at "being cancelled", now has an empty Youtube channel (graduated)
• Thin tweeted a google doc confirming that he was the owner of the hate chat, but the origin was not meant to be a hate chat
• Phoenixx tweets of knowing of Dove doxxing and claims he held Dove responsible, and didn't know of a hate group chat. Kenji says Phoenixx was the one to tell him that Dove wasn't sorry/downplayed the dox. Later on in a VOD, Kenji seems to be on bad terms with Phoenixx
• RiiKami tweets a google doc saying he "wasn't in a group"(supposedly either group) during "when this big thing happened"
• AndySSJ apologized and I personally don't know anything about his involvement, besides being a Mofongo Boy
• Kage was in the hate groupchat with Dove, Thin, and others. He and Kenji are on bad terms.

I’ve recently been testing out an Intune deployment and finding app installations failing sometimes with the error code 0x80d02017.
Further investigation showed that sometimes the Microsoft Store couldn’t be opened with the error “Sorry about that! Something went wrong, but we are making it right. Try refreshing or come back later.”. If the store did load, it failed to install apps/updates with the error “something happened on our end”.
I checked out my Intune configuration and GPOs but nothing set to block the store.
Checked the firewall and found connections to a set of IPs coming from all of our Windows machines being blocked. We block all explicit IP address connections (1.2.3.4) on port 80 & 443 that do not use a DNS name (url.website.com) unless explicitly whitelisted.
Anyway it turns out that the range 151.139.43.0/24 is being used to push Microsoft store content and app updates. They are not using a DNS name, they are connecting direct by IP. By unblocking this range, the store behaves like normal and Intune app deployment works.
Usually updates come from URLs and they’re much easier to manage/categorise, so it’s very frustrating that Microsoft are pushing updates via IP addresses. I hope this helps someone else facing the same problems, as I couldn’t find anything online about this IP address range.

Got the free trial for Armor with my Orbi. Worked fine for a couple of weeks, then today I started getting odd issues when accessing my company's network remotely. Noticed a couple of URLs getting repeatedly blocked starting today - manually unblocked them and still had issues. Finally turned Armor off completely and everything resumed working as expected.
This is a scenario where one needs to be able to disable Armor on specific devices on the network. (Before someone says to 'just uninstall it on that computer' - there are no client bits - it runs server side and is controlled by the router for all devices on the network,)

Hello.
We have recently rolled out NinjaRMM across our endpoints and have seen a lot of failures to install 3rd party patches (things like FileZilla, Notepad++, WebEx, Chrome, etc).
I've already overcome a few issues (we're behind Sophos SG firewalls, and by default all web traffic is blocked, unless proxied, so I've unblocked thousands of Amazon IP addresses). I've also added the URLs to the exclusions for web filtering. These are the hosts I've used:
​

• agent-app.ninjarmm.com
• connect-us-west-s2.ninjarmm.com
• resources.ninjarmm.com

Now, in the logs, I'm seeing HTTP 401 errors. As the log is really verbose, and consists of thousands of lines, here's what stands out:
​
SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
ALPN: server accepted http/1.1
Server certificate:
subject: CN=*.ninjarmm.com
start date: Jan 30 00:00:00 2024 GMT
expire date: Feb 27 23:59:59 2025 GMT
subjectAltName: host "agent-app.ninjarmm.com" matched cert's "*.ninjarmm.com"
issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M03
SSL certificate verify ok.
using HTTP/1.1
<-HTTP/1.1 401 Unauthorized
<-Date: Thu, 04 Apr 2024 05:53:53 GMT
<-Content-Type: application/json
<-Content-Length: 135
<-Connection: close
<-Server: nginx/1.21.5
<-Vary: Accept-Encoding
<-
Closing connection
TLSv1.2 (OUT), TLS alert, close notify (256):
CURLINFO_HTTP_CODE=401
CURLINFO_HTTP_CONNECTCODE=0
06:53:53.294 E 3PPWRK:1876 [ serverresponse.cpp:73] RspSize[135] StatusCode[401] RequestURL[agent/credential] ResponseCode[FAILURE] ErrorInfo[HTTP 401 Unauthorized]
06:53:53.294 E 3PPWRK:1876 [ distressmonitor.cpp:293] Distress Signal[FAILED_HTTP_POST_FETCH_CREDENTIALS]:(5) Got error 401 when POST
06:53:53.294 E 3PPWRK:1876 [ credentialmanager.cpp:123] Won't retry to fetch credentials...
06:53:53.294 E 3PPWRK:1876 [ ProcessUtils.cpp:85] Failed to retrieve credentials from server.
​
Has anybody else seen anything similar?

Strategies for Writing Unblockable Web Scrapers in Python

In the realm of web scraping, one of the biggest challenges developers face is the constant threat of being blocked by websites. However, there are strategies you can employ to enhance the longevity and effectiveness of your web scrapers. In this blog post, we'll delve into five key strategies for writing unblockable web scrapers in Python.

User-Agent: Mimicking Human Behavior

The User-Agent header plays a crucial role in convincing servers that your scraper is a legitimate user. By setting a User-Agent that mimics different browsers and devices, you can avoid detection and prevent blocking. Utilizing a diverse pool of User-Agents enhances the realism of your requests and reduces the likelihood of being flagged as a bot.

Implementation:

• Utilize libraries like requests to set custom User-Agent headers.
• Randomize User-Agents from a predefined list to emulate human behavior effectively.

Referers: Directing Traffic Ethically

Setting Referer headers helps establish a connection between the current page being scraped and the previously visited page. By providing genuine Referer information, you can simulate natural browsing behavior and evade suspicion. Analyzing website structures and utilizing relevant category URLs or genuine backlinks as Referers enhances the credibility of your requests.

Practical Approach:

• Employ tools like SEMRush to identify relevant backlinks for specific domains.
• Dynamically set Referer headers based on the page being scraped to mimic authentic user interactions.

Proxy IPs: Diversifying Your Presence

Using proxy IPs is paramount for avoiding IP-based blocking. By rotating through multiple proxy IPs, you can distribute requests and mitigate the risk of detection. Investing in reputable proxy services that offer diverse IP pools ensures resilience against blocking attempts based on IP patterns.

Best Practices:

• Procure multiple proxy IPs to establish a robust infrastructure for your scrapers.
• Regularly rotate proxy IPs to prevent detection and maintain anonymity.

Request Headers: Mimicking Authentic Requests

Incorporating essential request headers beyond User-Agent and Referer further enhances the authenticity of your scraper's requests. Analyze the headers sent by genuine browsers and replicate them accurately to avoid detection. By including commonly accepted headers such as Accept-Encoding and Accept-Language, you align your scraper with typical browsing behavior.

Implementation Tips:

• Scrutinize network requests using browser developer tools to identify additional headers.
• Customize request headers based on specific website requirements to ensure compatibility and authenticity.

Delays: Emulating Human Interaction

Introducing delays between successive requests is crucial for emulating human browsing patterns. By incorporating random delays, you prevent triggering rate-limiting mechanisms and evade detection. Implementing varied delay intervals reduces predictability and enhances the stealthiness of your scraping activities.

Practical Considerations:

• Utilize libraries like numpy or random to introduce random delays between requests.
• Experiment with different delay distributions to strike a balance between efficiency and realism.

Conclusion: Striking a Balance

While achieving complete immunity from blocking may be challenging, implementing these strategies significantly increases the resilience of your web scrapers. By mimicking human behavior, diversifying your presence, and optimizing request patterns, you can navigate through the web's defenses more effectively. Continuous refinement and adaptation are key to maintaining successful and unblockable web scraping operations.

Meta Description:

Learn effective strategies for writing unblockable web scrapers in Python. Enhance your scraping prowess and evade detection with expert insights and practical tips.

HASHTAGS:

- online data scraping tools free
- scrapoer api

​
I am so frustrated. How to unblock all these requests. I tried adding the subdomains to Trust Lists, and I don't how about My Rules, But I messed there too. The video direct url opens in a new tab, but doesn't open when invoked by the website

We have a Linksys Velop system, MR8300v1 with 2 nodes (WHW03v2). We have 2 kids with ADHD and rely heavily on the parental controls.
Lately, they are just crap. I will choose to block internet and it will give me an error "settings not changed" and nothing I can do will work aside from rebooting the whole network. Sometimes the settings "work" on the router, but the actual device itself doesnt get the settings. So despite internet and YouTube being blocked, the kids can continue to use the device unless I go and power the device on and off.
And in reverse, devices will sometimes remain blocked even though they have been unblocked.
We tried static IPs to see if that somehow was the issue (locking a device whose IP changed) but the issue persists.
We tried using time blocked settings but those are crap, hence why I'm manually going into the Linksys software all day long to lock and unlock devices.
We want YouTube 100% blocked in our house as these kids become zombies when it's on. But we found out that Google patching and Google search is affected if we block the YouTube url. So we have to unblock it to use devices as an adult.
Oh, and the node names never stick!! So I can set it up so I know which device is which, but then 24 hours later they all lose their names and I don't know which is the TV or a phone or a laptop to block. Some devices (like our printer), the name never changes. But their stupid chrome books, the name is always different every day and I have no idea why.
All this used to work so well when we first got the devices. Not it's a huge pain because ether software never works.
Any ideas on how to fix it or improve? Static or DHCP? Specific firmware version that is more stable (we have them on the latest). Other setting such as mac address filtering?

Hi All, i try to found where i have an issue for my nextjs applicaiton. Aim is to have logged user with role "ADMIN" and do update for user block unblock. My problem is that from client to server is not passing session and receiving error:
using nextauth, nextjs, prisma
https://next-auth.js.org/errors#client_fetch_error undefined {error: {},url: 'http://localhost:5002/api/auth/session',message: undefined}
i tried to changed localhost to dns server (url) but it not changed nothing. if i try api/auth/session directly from browser is give me session with all information.
here is my code for block user:
import { NextApiRequest, NextApiResponse } from "next"; import { getSession } from "next-auth/react"; import { PrismaClient } from "@prisma/client"; import { BlockUserRequestBody, BlockUserResponse } from '~/types/api'; // Adjust the import path based on where you placed your interfaces const prisma = new PrismaClient(); export default async function handler( req: NextApiRequest, res: NextApiResponse ) { if (req.method !== "POST") { return res.status(405).end(`Method ${req.method} Not Allowed`); } const session = await getSession({ req });
if (!session session.user.role !== "ADMIN") { return res.status(403).json({ message: "Unauthorized" }); } const { userId } = req.body as BlockUserRequestBody; try { await prisma.user.update({ where: { id: userId }, data: { blocked: true }, }); return res.status(200).json({ message: "User blocked successfully" }); } catch (error) { console.error("Failed to block user:", error); return res.status(500).json({ message: "Internal server error" }); } } if i will remove condition if (!session session.user.role !== "ADMIN") { return res.status(403).json({ message: "Unauthorized" }); } it works fine so problem is with identifiing session.
here is my manage-users componnent:
// pages/admin/manage-users.tsx import React, { useState, useEffect } from 'react'; import { useSession, getSession } from 'next-auth/react'; import { useRouter } from 'next/router'; import Layout from '../../layouts/Layout'; // Adjust the import path as needed import UserTable from '../../components/users/UserTable'; // Adjust the import path as needed import UserDetailModal from '../../components/users/UserDetailModal'; // Adjust the import path as needed import { GetServerSideProps } from 'next'; const ManageUsers: React.FC = () => { const { data: session } = useSession(); const router = useRouter(); const [selectedUser, setSelectedUser] = useState(null); const [users, setUsers] = useState([]); useEffect(() => { console.log(session); // Log the session data if (!session session.user.role !== 'ADMIN') { router.push('/'); // Redirect non-admins to homepage } else { fetchUsers(); // Fetch users if the user is an admin } }, [session, router]); const fetchUsers = async () => { const res = await fetch('/api/users'); const data = await res.json(); setUsers(data.users); }; const handleEdit = (user) => { setSelectedUser(user); }; const handleCloseModal = () => { setSelectedUser(null); // Reset selected user }; const handleSaveUser = async (updatedUser) => { // API call to update user details here // Refresh the user list or update state as needed here setSelectedUser(null); // Reset selected user after saving fetchUsers(); // Fetch updated user list }; return (

Manage Users

{selectedUser && ( )}

); }; export default ManageUsers; export const getServerSideProps: GetServerSideProps = async (context) => { const session = await getSession(context); if (!session session.user.role !== 'ADMIN') { return { redirect: { destination: '/', permanent: false, }, }; } return { props: {} }; };
and userTable.tsx component:
import React, { useEffect, useState } from 'react'; interface User { id: number; name: string; email: string; role: string; emailVerified: Date null; blocked: boolean; } const UserTable: React.FC = () => { const [users, setUsers] = useState([]); const [loading, setLoading] = useState(true); const [error, setError] = useState(''); useEffect(() => { const fetchUsers = async () => { try { setLoading(true); const response = await fetch('/api/users'); const data = await response.json(); setUsers(data); } catch (err) { setError('Failed to fetch users'); console.error(err); } finally { setLoading(false); } }; fetchUsers(); }, []); const toggleBlockUser = async (userId, blocked) => { try { const response = await fetch(`/api/users/block`, { method: 'POST', headers: { 'Content-Type': 'application/json', }, credentials: 'include', // Ensures cookies are sent with the request body: JSON.stringify({ userId: userId, blocked: !blocked, // Toggle the block status }), });
if (!response.ok) { throw new Error(`Error blocking user: ${response.statusText}`); }
// Assuming the API returns the updated user object or some confirmation const data = await response.json(); console.log('Block user response:', data); // Handle success - perhaps update local state or UI accordingly } catch (error) { console.error('Error blocking user:', error.message); // Handle error - update UI or state to reflect the error } };
if (loading) return

Loading...

; if (error) return

Error: {error}

; return (
{users.map(user => ( ))}

ID	Name	Email	Role	Verified	Blocked	Actions
{user.id}	{user.name}	{user.email}	{user.role}	{user.emailVerified ? 'Yes' : 'No'}	toggleBlockUser(user.id, user.blocked)} />	{/\* Placeholder for Edit button \*/}

); }; export default UserTable;
any advise what could be wrong.? I am able to log with admin role and have authentificated user. Thanks so much

Hello,
​
I'm working with a Fortigate-VM, Firewall, and I'm having issues whitelisting a certain url...
In the past, I've had no issues creating a Web Filter>URL Filter entry with , type:simple, action:exempt.
​
However, with this site, normal entries in the style of the above example don't work. So I've tried different variations, including but not limited to wildcards and ip addresses.. Seemingly no matter what I do, when this machine attempts a call, I get another blocked entry in my web filter logs..
​
Details:>Call is attempting to make a connection over HTTPS on port 443.>I'm in the US, target destination is also in the US.>From the Web Filter logs "Message URL belongs to a denied category in policy" -This is true, but I'm not trying to unblock the whole category just for this one url. That's why I'm attempting to whitelist said url...>I've tried using just the root (destination) domain of the url as both a simple and wildcard type, neither works.
​
Funny thing is, if I create a whitelist entry, and attempt to plug in the url in my browser on my local machine, it works. Well, I get an error, " Error code: SSL_ERROR_NO_CYPHER_OVERLAP," but the url otherwise appears valid. I get a "passthrough" entry in my web filter logs. Wut.
​
Thoughts?