For site owners
codai web agents
codai is an AI assistant with its own web search. Three automated agents make requests to websites.
They are operated by Interactive Media Solutions S.R.L. (brand codai), CUI 37237457, Reg. Com. J18/241/2017, Str. 23 August nr. 1B, Et. 1, Ap. 8, Târgu Jiu, Gorj, Romania. Contact: hello@codai.ro.
| Agent | User agent contains | What it does |
|---|---|---|
| Crawler | codai-crawler | Visits pages on a schedule to build the codai search index. |
| Search | codai-search | Fetches a page when someone’s search needs it, to check that the text we quote is really on the page, and checks now and then that indexed pages still exist. |
| Fetch | codai-fetch | Fetches one address because a person, or an AI agent working for them, asked to read that page. It does not follow links. |
How to control them
- robots.txt: use the tokens
codai-crawlerandcodai-search. Without a group for them, the*group applies. We reload robots.txt every 24 hours. - To keep a page out of the index:
<meta name="robots" content="noindex">or the headerX-Robots-Tag: noindex. - Text and data mining reservation (Directive (EU) 2019/790, Article 4(3)): we honour the header
tdm-reservation: 1and<meta name="tdm-reservation" content="1">. Such pages are not stored. codai-fetchacts on a direct request from a person, like a browser, and does not read robots.txt.
How we crawl
- The crawler makes at most one request per second to a host by default and follows
Crawl-delaybetween 1 and 10 seconds. - The crawler reads HTML pages only, up to 3 MB, 10 seconds per request, no cookies, no forms, no login.
- We use conditional requests, so unchanged pages are not downloaded again.
What we keep and show
- The index stores the readable text of a page (up to 60,000 characters), its title, address and dates.
- Search results show the title, the link and a short quote of up to 300 characters. API users can ask for a longer excerpt of up to 2,000 characters.
- Results are given to people and to AI agents as sources, always with the link to your page.
Removal and personal data
- To remove a page or a site from the index, write to hello@codai.ro with the address. We remove it and block it from being added again.
- If a result shows personal data about you and you want it removed, write to privacy@codai.ro. We answer within one month. You can also complain to the Romanian data protection authority (ANSPDCP).
- To report illegal content: use the notice form.
How to recognise us
Our requests come from Google Cloud (region europe-west1). We do not have a fixed list of IP addresses yet, so please identify us by the user agent. If you see behaviour that does not match this page, write to hello@codai.ro and include the time and address.