Disable delete optimization and exit ref loop faster

[ddanielr]

This PR adds a property to skip scanning the metadata table for table deletes.

DeleteMarker Creation Optimization

When the manager deletes a table it performs an optimization step by creating a batch scanner with 8 threads (not configurable) to scan all the other table file references on the metadata table and ensure that no file references are found for the given table volume.

The manager then directly deletes the volumes as opposed to writing delete markers and allowing the GC to handle the tablet file deletions.

This is a nice optimization to have when dealing with a small static set of tables. However, when table creation is dynamic these scans can cause unnecessary delays and/or hanging fate processes as all metadata tablets must be scanned in order to process a single table delete.

Unnecessary File Ref Counting

The batch scanner only needs to produce a single shared file ref result (refCount) in order to trigger delete markers to be created as the code only checks if refCount is equal to zero.

However, the existing code needlessly counts all of the found refs first.
This is unnecessary and a fast break was added to the iterator loop for the batch scanner.

[ddanielr]

@keith-turner & @ctubbsii I'm also leaning towards just removing this delete marker optimization entirely.

I wanted to be a bit more careful about it in 2.1 but if you don't see a reason to keep this code around then I'll push a change to just remove the scan and volume deletion shortcut entirely.

[keith-turner]

I wanted to be a bit more careful about it in 2.1 but if you don't see a reason to keep this code around then I'll push a change to just remove the scan and volume deletion shortcut entirely.

I support removing this optimization in general. Do share the concerns about introducing bugs in 2.1. Looked at what test we have for delete table to see if any verified the files actually eventually go away. I could not find any test that verifies this. I looked at test method names that called TableOperations.delete and for test I was not sure about based on the name I looked at the test.

Maybe part of removing this in 2.1 is adding some more test. Want to ensure the files exists before delete and that after delete the tables file and dir eventually go away. That would cover the risk that delete table leaves files behind w/ this change. If delete does more than its supposed to (like messes w/ another table) I suspect existing test would catch that as we create/delete a lot in the test.