Allow Command

The robots.txt directive that permits crawling of a path, typically to carve an exception out of a broader block.

Its normal use is narrow: disallow a directory, then allow the one file inside it that must stay crawlable.

The longest matching rule wins, not the first one, so test every change in the robots.txt tester before shipping it.

Without it, blocking a section would also block the assets and pages inside it that you still need crawled.

Allow rules are where robots.txt mistakes hide, because they look harmless and quietly override a block.