1
1
---
2
- date : 2022-02-23
2
+ date : 2022-04-13
3
3
title : " FileSystem Listing"
4
4
linkTitle : " FileSystem Listing"
5
5
weight : 30
@@ -115,7 +115,7 @@ The `IgnoreHiddenFileFilter` can be used to filter hidden files from being read.
115
115
** Configuration example**
116
116
117
117
``` properties
118
- fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.scanner.local .filter.IgnoreHiddenFileListFilter
118
+ fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.fs .filter.IgnoreHiddenFileListFilter
119
119
```
120
120
121
121
{{% alert title="Limitation" color="warning" %}}
@@ -124,20 +124,30 @@ This filter is only supported by the `LocalFSDirectoryListing`.
124
124
125
125
### LastModifiedFileFilter
126
126
127
- The ` LastModifiedFileFilter ` can be used to filter files that have been modified to recently based on their last modified date property.
127
+ The ` LastModifiedFileFilter ` can be used to filter all files that have been modified to recently based on their last modified date property.
128
128
129
129
``` properties
130
- fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.scanner.local .filter.LastModifiedFileFilter
130
+ fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.fs .filter.LastModifiedFileFilter
131
131
# The last modified time for a file can be accepted (default: 5000)
132
132
file.filter.minimum.age.ms =10000
133
133
```
134
134
135
135
### RegexFileFilter
136
136
137
- The ` RegexFileFilter ` can be used to filter files that do not match the specified regex.
137
+ The ` RegexFileFilter ` can be used to filter all files that do not match the specified regex.
138
138
139
139
``` properties
140
- fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.scanner.local .filter.RegexFileListFilter
140
+ fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.fs .filter.RegexFileListFilter
141
141
# The regex pattern used to match input files
142
142
file.filter.regex.pattern =" \\.log$"
143
+ ```
144
+
145
+ ### SizeFileListFilter
146
+
147
+ The ` SizeFileListFilter ` can be used to filter all files that are smaller or larger than a specific byte size.
148
+
149
+ ``` properties
150
+ fs.listing.filters =io.streamthoughts.kafka.connect.filepulse.fs.filter.RegexFileListFilter
151
+ file.filter.minimum.size.bytes =0
152
+ file.filter.maximum.size.bytes =9223372036854775807
143
153
```
0 commit comments