You can limit the number of Parquet files to profile in non-hive partitions to 10 in Amazon S3 V2 scanner. Use the following custom option in the Metadata Load Settings tab of the resource to enable limited profiling:
-DExecuteMetaParquet=true
To limit the number of files, ensure that
Disable Partition Pruning
in the Metadata Load Setting tab is unchecked. If you disable partition pruning, all the Parquet files in non-hive partitions will be profiled.
For more information, see the
Enterprise Data Catalog 10.5.5 Scanner Configuration Guide