What ABFS Vectored Read does
ABFS is the Azure Blob File System driver, the Hadoop-compatible connector for Azure Data Lake Storage. Microsoft's documentation says ABFS is part of Apache Hadoop and is included in many of the commercial distributions of Hadoop. Frameworks such as Spark and Presto use it to reach data lake storage.
A Microsoft Tech Community post describes the new capability. It helps compatible analytics engines reduce request overhead when reading selected portions of columnar files. The mechanism, in the post's words, works like this: a compatible reader submits all the file ranges it needs together. ABFS validates and sorts the ranges, combines nearby ones where appropriate, and splits the work to fit its read buffers. It then queues the storage requests asynchronously and returns the bytes to the matching range buffers.
The post is explicit that this is a client-side optimization. It also says it does not change the file format or rewrite your store. In practice, nothing about your stored data or its layout changes.
Why selective reads benefit
A selective Parquet query might need a file footer, a few column chunks and some metadata. Those pieces are scattered, not contiguous. The Tech Community post explains the cost: each request carries network and processing overhead, so selecting less data doesn't necessarily remove the cost of coordinating those reads. Handing the ranges over together lets the client plan the I/O as a group. It can then merge neighbouring ranges and issue fewer requests.
The underlying Hadoop interface is readVectored(). Apache's API documentation says it reads a list of file ranges asynchronously. Each range gets a future that completes with a ByteBuffer holding that range's data. The documentation also warns that the default implementation reads each range synchronously, with the intent that stream subclasses provide more efficient versions. The ABFS work is such an implementation.
What you need for it to work
The benefit isn't automatic. Microsoft's post sets two conditions: the Parquet reader must support vectored I/O, and the runtime must include a compatible ABFS client. Spark workloads using higher-level APIs can then enable the capability through configuration, with no query rewrites, application logic changes or data migration.
Apache's current ABFS configuration reference, for the 3.6.0-SNAPSHOT development line, lists keys that control the behavior:
fs.azure.vectored.read.strategyselects the vectored read strategy ABFS uses.fs.azure.min.seek.for.vectored.readssets the minimum gap between adjacent ranges when merging.fs.azure.vectored.read.max.merged.sizesets the maximum merged size for ranges.fs.azure.vectored.read.max.merged.size.throughputis the equivalent for the throughput-optimized mode.
The reference describes what each key is for. It doesn't list accepted values or defaults. It also isn't a release compatibility matrix. Don't copy settings from a snapshot reference into a production cluster without checking your own distribution's documentation.
Practical checklist for data platform teams
- Check your runtime. Confirm that your Spark or Hadoop build ships an ABFS client with vectored read support. A snapshot API page doesn't tell you what your vendor ships.
- Check your reader. The Parquet or ORC reader has to call the vectored API. A compatible driver alone does nothing if the reader doesn't use it.
- Test on a representative workload. Gains should show up where queries touch many small, separated ranges. They are less likely for full sequential scans. This is general reasoning, not a measured result.
- Tune carefully. Merge gap and merged-size settings trade request count against bytes transferred. Change them one at a time and measure.
- Keep rollback simple. Because it's configuration-driven, you should be able to compare runs with the feature on and off.
API caveats
Hadoop's interface documentation lists general constraints. These are API-level cautions, not new ABFS-specific restrictions:
- Overlapping or invalid ranges raise an
IllegalArgumentException. - The stream position after the call is undefined.
- Normal read calls may block while a vectored read is in progress.
- If the file changes during the operation, the output is undefined, and ranges may mix old and new data.
What remains unknown
The Azure Updates notice doesn't give speedup figures, billing effects, regional limits, or minimum client versions. The Tech Community post describes the mechanism and the benefit in general terms. This article doesn't present any performance number, and you shouldn't assume one. The post is also still new, so expect documentation and vendor runtime notes to fill in over the coming weeks.
Bottom line
This is a modest but useful change for people running selective analytics against Azure Data Lake Storage. The gain comes from coordinating reads on the client, and it depends on your reader and runtime supporting vectored I/O. If you run Spark on Parquet in Azure, check your runtime's ABFS version and benchmark the feature on real queries before changing production defaults.
References
- (Launched) Generally Available: ABFS Vectored Read for Azure Data Lake Storage Azure Updates · 2026-10-22T18:01:00Z
- Apache Hadoop Azure support – Hadoop Azure Support: ABFS - Azure Data Lake Storage Gen2 hadoop.apache.org
- Optimize selective analytics reads in Azure Data Lake Storage with ABFS Vectored Read techcommunity.microsoft.com