hi, no worries at all, it came off perfectly fine

yes, we use residential proxies + all requests go are js-rendered, we maintain a caching layer which is 95%+ opted into by customers

it's all included in the credit price, great value compared to alternatives, the business model does rely on scale and our margin gets better the more requests we serve (esp infra cost for k8 + browser fleet)

to answer your e.g., yes public linkedin pages will work fine, anything behind a login we don't really support out of the box until we can figure out a safe way to do so, since that's where red lines are drawn

we step in whenever we see our service is hitting a website more than it should, this usually means reaching out to a customer for clarity on why they are not opting into the cache, we have alot of safeguards around fraud/spam and will let someone know if their request pattern looks like they're causing harm

hey, happy you like it, to be honest we had no real secret, it takes a ton of time and effort to earn developer trust

i got my first customer after 1 month, and then just worked like hell to make him happy even though it was a very large company paying me $99/mo at the time, they're still a customer too

since then, i just locked in on every customer until they were thrilled, and once we hit 50, word of mouth started to kick in, now we're at 300+ and get a referral every other day which is awesome

i think our community appreciates effort, and when someone goes the extra mile to make sure your problems are solved, you tend to remember that. it doesn't scale super well, but you'd be surprised how far it'll take you

oh wow, happy to see you here, we didn't post this anywhere so its nice to see a customer find us so quickly, thank you for the kind words.

brand data is a shockingly hard problem to get right

oh also on knifegeek, it'll be live again this weekend when i find a moment to fix the bug causing it to crash, i collect knives/watches so it was a super fun project to work on

we try to read robots.txt, it's definitely more efficient if agents just see what humans see, we run a custom browser rendering stack

in terms of llms.txt, we're not primarily an AI product (although some features do use LLMs), and speaking to friends who run products it seems to be not very helpful, even though we have one as well, i didn't see it move the needle much

even my own coding agents don't look at llms.txt when looking at our own website, so unsure of whether that standard will survive the test of time

happy you noticed that, i put quite a bit of love into the ui, i've found engineers care alot about polish & feel of the webapps they use, even for an api product

i'm an engineer by trade, and always hated things like forcing a sales call, or having hidden credit multipliers, i tried to build this with the same ethos i like for my own dependencies (shoutout axiom.co)

im not sure the models do this, rather the tooling around them, however web search/extraction by the model providers is ridiculously expensive and quite slow, so going into production it makes sense to use a provider (like us)

will let my designer know, it's a dev-focused product, and we're all so finely tuned to avoid slop, so design & feel was really important

thank you, i try to ship updates often: https://docs.context.dev/changelog

There's alot of overlap at the moment, but we're diverging quite quickly.

If you want differences as of right now

- 1 credit = 1 scrape, no hidden credit multiplier - we have world class brand data - we're focused primarily on the infra use-case, rather than gtm & everything else - anecdotally, customers have seen their error rates drop quite dramatically

In general it's a huge space, firecrawl is a wonderful company, it's fun to compete with them, planning to add more things soon which should make the differences clear

yes but we do alot more that may not be clear at first glance, things like brand data, and for scraping handling pdf, ocr, docx, ppt, xlsx automatically

shipping a bunch of new things soon which should make it clearer, but as of today yeah

you're welcome to try it on our demo page (no signup needed), should handle everything just fine, yes we don't mention ips on the homepage

also, while it might seem expensive, we're cheaper than every other option out there, because there's no hidden credit multipliers. every single customer who uses us halved their bill + error rate

im not sure if grok is on here, i think that's an X thing

but if you were to ask me, we're more fully featured than cloudflare, and anecdotally a ton more reliable in terms of error rates. back when it was brand.dev, i actually tried to use cloudflare's apis and it was quite unreliable, so we built our own stack instead

Maybe AI-service-blocklist-as-a-service could be a YC company.

Interesting - I'll be sure to benchmark it at some point. We've found the best results come from blending providers depending on the task anyways.

Thanks for the quick response - and always happy to see more competition in the space. Best of luck with future features!

Better business model would be some sort of micropayment setup - allow humans, but make scrapers pay a hundredth of a cent for access.

please do, and if you see anything off let me know, we've yet to lose a single "bakeoff", i normally ask customers to just have cc/codex run it so its somewhat unbiased

on the 2nd point, most industries are not zero-sum, and many of customers use multiple data providers in any case, so agree with you there

thank you!

Is this similar? https://blog.cloudflare.com/monetization-gateway/

Yep. It will be interesting to see if things move that way or not. AI scraping might get annoying but a lot of sites (eg e-commerce) have obvious incentives to allow AIs to scrape them for free (or even pay AI companies to prioritize them, like they already do with Google).