URL / Legislation Ingestion

Paste the link of a law or of a case-law corpus and DATTA delivers the articles, the related decisions and semantic search ready to use — without downloading a single file. Copying a code article by article into a system takes days, and it still arrives out of date. The URL tab of the Upload page does it in the background, with a real progress bar: you provide the address, choose the target context and can leave the screen without interrupting the load.

What URL ingestion does

The tab accepts a web address and decides on its own how to process it. When the content is legislation (for example, a law from Planalto or a page from ENFAM's corpus927), the platform:

• splits the text into individual articles;
• enriches each article with the related case law (court decisions) extracted from corpus927;
• writes the articles and the decisions to the context's graph, with the appropriate relationships;
• indexes the full text and the semantic vectors in the context's search index, so everything shows up in Search;
• identifies each article's entities with the model configured for the context.

The result: the loaded legislation becomes searchable both by literal keyword and by meaning, with the judicial decisions already linked to each article.

The same tab also knows how to handle other URL types (regular HTML pages, downloadable CSV/ZIP files, file folders, CNPJ data from the Receita Federal). This guide focuses on the legislation flow, which is the richest. The other types are detected automatically and processed through their own paths — you do not need to choose the type.

Before you start

Step by step

1. Open the Upload page

Go to Processar›Upload. The breadcrumb at the top confirms where you are: DATTA › Processar › Upload.

2. Choose the target context

In the Contexto de Destino block, select the context where the legislation should be written. The active context is highlighted right below ("Contexto ativo: ..."). Without a selected context you cannot proceed.

3. Select the URL tab

Right below the context selector there are three tabs: PDF, URL and Texto. Click URL.

4. Paste the link

In the Upload de URL Web field, paste the address. Typical examples:

• A Planalto law — e.g. the Code of Civil Procedure (https://www.planalto.gov.br/.../l13105.htm).
• A corpus927 (ENFAM) page — for legislation already accompanied by case law.
• Other pages of norms (e.g. Normative Instructions from the DOU, https://www.in.gov.br/...).

The field accepts pressing Enter to submit.

5. (Optional) Filter articles

The Filtro de artigos field lets you load only a subset — useful for a quick test before processing an entire law:

Leave the field empty to load all articles.

6. (Optional) Refresh the entire base

In contexts of a legal nature the Atualizar toda a base option appears. When checked, it removes the context's existing articles, paragraphs, items and subitems and reinserts everything from the new ingestion (the semantic vectors need to be regenerated afterwards). Use with caution — the interface shows a red warning before confirming.

7. Start

Click Carregar URL. The ingestion starts in the background and the progress bar appears immediately; the button switches to "Iniciando..." while the task is being created.

What happens after you click

The platform creates the task, immediately returns a task identifier (taskId) and keeps processing in a decoupled way — the interface is not "stuck" waiting for the end. Automations can trigger and follow the same ingestion; see the API reference.

For legislation, processing goes through five phases, reflected in the progress bar:

Three points worth noting:

• The model is never fixed in code. Both the semantic vectors and entity recognition use the platform's active model, defined in the context settings. Changing the model there changes what is used in the next ingestion.
• Case law depends on the legislation being recognized. For the mapped codes (CPC, Civil Code, CDC, Constitution), the platform locates the correct page in corpus927. If the legislation has no known match, the articles are written normally and the case-law phase is skipped with a warning.
• Each kind of content goes to the right place. The long text and the vectors go to the search index; the graph keeps only light metadata (article number, link code and a short excerpt) and the relationships.

Tracking and cancelling

• Real progress. The interface checks the task state about every 2 seconds and updates the bar with the percentage and the current phase's message (e.g. "Jurisprudência Art. 970 (5/12)").
• Survives leaving and coming back. The task identifier is stored locally in the browser. If you navigate to another area and come back to the URL tab, the interface checks the task's real state and resumes tracking automatically — but only if it is still in progress. If it has already finished (or no longer exists), the interface clears the leftover state and stays idle, without showing false progress.
• Cancel. During processing there is the Cancelar button. At the end (success, error or cancellation) a summary appears together with the Ingerir outra URL button to start over.

Processing runs on the server even with the tab closed. Closing the page does not cancel the ingestion — to actually stop it, use Cancelar.

The target context does not change mid-load

Each ingestion writes to one specific target context, chosen at the moment you click Carregar URL. That target is pinned for the entire load: once started, the ingestion keeps writing to the original context until the end, regardless of what you do on the screen afterwards.

To make this visible, the progress block displays a fixed banner while the load is in progress:

• Upload em andamento — indicates there is an active ingestion.
• Contexto de destino: {contexto} — shows, prominently, exactly which context the articles, vectors and entities are being written to.

The banner keeps the same context until the task's final state (completed, failed or cancelled). The completion message also cites the load's real target context, not whichever context is active on the screen at that moment.

Why this matters. You can start an ingestion in one context and then click another context in the top chips to start preparing another operation. Previously, that switch made the whole screen appear to reflect the new context — giving the false impression that the in-progress load had changed its target. The real target never changed; now the screen makes that clear as well.

Warning when you switch contexts

If, with a load in progress, you select a context different from the ingestion's target context, the interface shows a warning right below the banner:

"A troca de contexto acima não afeta esta carga: ela continua sendo gravada no contexto de destino indicado."

Switching contexts during an upload is allowed on purpose — it lets you start preparing the next operation in another context without waiting for the current load to finish. The warning simply confirms that the ongoing ingestion remains intact in its original target.

When leaving and returning to the page

When you navigate to another area and return to the URL tab, tracking is resumed (see the previous section). On that return, the target context shown in the banner comes from the task's own state on the server — that is, it is always the correct context the load is writing to, even if the active context on the screen has been changed in the meantime.

Old loads started before this improvement may not have the target context recorded; in those rare cases the banner falls back to the active context for display only. The write always happened to the correct target.

Reprocessing without duplicating

Ingestion is idempotent: resubmitting the same URL does not create duplicate content.

• Legislation. Articles are written by merge in the graph — reprocessing the same law updates the existing nodes instead of duplicating them. The deduplication by article number described above ensures each article enters only once even when the page repeats the same "Art. N" in nested markup.
• PDF. When the content is a PDF file, deduplication is by sha256: the document code derives from the file hash, so resubmitting the same PDF overwrites the same record (in the graph and in the index) instead of creating a copy.

In short: you can reprocess at will to fix or complete a load — the database does not bloat.

Tips

• Small test before the full load. Use the Filtro de artigos field with a short range (e.g. 966-970) to validate that the URL is recognized and that case law is coming through. Then run without the filter for the full load.
• Large laws take time. The case-law phase makes one query per article, with a short pause between them so as not to overload the source. For a code with many articles this can take quite a while — the progress bar (the 15% to 95% range) is the best indicator of progress.
• Check the result in Search. As soon as the indexing phase finishes, the articles already show up on the Search page (literal + semantic text), with the decisions linked to each article.
• Let it run. Since processing survives navigation, start the ingestion and keep using other areas of the platform; come back to the URL tab whenever you want to check progress.

Troubleshooting

On any failure, the interface shows the message in Portuguese, without exposing raw error codes, stack traces or technical diagnostic data.

See also

• Search — where the ingested legislation becomes searchable.
• AI-powered screening rule generation — uses the articles and case law loaded here as the grounding for the rules.
• API reference — to automate triggering and tracking the ingestion.