Skip to content

feat!: accept pre-configured import components as options instead of options for components #283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 16, 2023

Conversation

achingbrain
Copy link
Member

@achingbrain achingbrain commented Feb 10, 2023

The existing importer takes one big options object that contains options for every subsystem. This leads to situations where different components use the same option keys - for example both the rabin and fixed-size chunkers use the maxChunkSize key.

This PR reduces the number of options by accepting preconfigured instances of internal components - e.g. instead of passing '"rabin" for the chunker argument, you pass an instance of the rabin chunker instead. This greatly simplifies the config and also allows a simple way for users to pass custom chunkers in. The same thing applies to file layouts, dag builders etc.

Before:

import { importer } from 'ipfs-unixfs-importer'

for await (const result of importer(stream, blockstore, {
  chunker: 'rabin',
  avgChunkSize: 512000,
  strategy: 'trickle'
}) {
  // ...
}

After:

import { importer } from 'ipfs-unixfs-importer'
import { rabin } from 'ipfs-unixfs-importer/chunkers'
import { trickle } from 'ipfs-unixfs-importer/layouts'

for await (const result of importer(stream, blockstore, {
  chunker: rabin({
    avgChunkSize: 512000
  }),
  layout: trickle()
}) {
  // ...
}

Other breaking changes:

  • This module is now typescript
  • CID 1 is now the default
  • Raw leaves: true is now the default
  • The onlyHash option has been removed - just pass a MemoryBlockstore or other blockstore implementation that doesn't store the blocks anywhere
  • The strategy config key is now layout and accepts an instance of a file layout instead of a string
  • The chunker config key now accepts an instance of a file chunker instead of a string

importFile, importDirectory, importBytes and importByteStream functions are also now exported along side importer to make dealing with simple imports that only import one thing easier.

import { importFile, importDirectory, importBytes, importByteStream } from 'ipfs-unixfs-importer'

// import single file
const result = await importFile({
  path: 'foo.txt',
  content: Uint8Array.from([0, 1, 2, 3, 4])
}, blockstore, options)

// import single directory
const result = await importDirectory({
  path: 'dir'
}, blockstore, options)

// import single buffer
const result = await importBytes(Uint8Array.from([0, 1, 2, 3, 4]), blockstore, options)

// import single byte stream
const result = await importByteStream(fs.createReadStream('./foo.txt'), blockstore, options)

BREAKING CHANGE: The options object now accepts preconfigured instances of chunkers and file layouts - these can be imported from this module - see #283 for more

Converts this module to typescript

Since there were existing types this should be minimally disruptive but
releasing this as a major just in case.

BREAKING CHANGE: the types on this module may have changed
@achingbrain achingbrain changed the title feat!: convert to typescript feat!: accept pre-configured import components as options instead of options for components Feb 13, 2023
@achingbrain achingbrain merged commit 5a38d01 into master Feb 16, 2023
@achingbrain achingbrain deleted the feat/convert-to-typescript branch February 16, 2023 07:18
github-actions bot pushed a commit that referenced this pull request Feb 16, 2023
## [ipfs-unixfs-v11.0.0](ipfs-unixfs-v10.0.0...ipfs-unixfs-v11.0.0) (2023-02-16)

### ⚠ BREAKING CHANGES

* The options object now accepts preconfigured instances of chunkers and file layouts - these can be imported from this module - see #283 for more

### Features

* accept pre-configured import components as options instead of options for components ([#283](#283)) ([5a38d01](5a38d01))
@github-actions
Copy link

🎉 This PR is included in version ipfs-unixfs-v11.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

github-actions bot pushed a commit that referenced this pull request Feb 16, 2023
## [ipfs-unixfs-importer-v14.0.0](ipfs-unixfs-importer-v13.0.0...ipfs-unixfs-importer-v14.0.0) (2023-02-16)

### ⚠ BREAKING CHANGES

* The options object now accepts preconfigured instances of chunkers and file layouts - these can be imported from this module - see #283 for more

### Features

* accept pre-configured import components as options instead of options for components ([#283](#283)) ([5a38d01](5a38d01))

### Dependencies

* update sibling dependencies ([b4f6fc8](b4f6fc8))
@github-actions
Copy link

🎉 This PR is included in version ipfs-unixfs-importer-v14.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

github-actions bot pushed a commit that referenced this pull request Feb 16, 2023
## [ipfs-unixfs-exporter-v12.0.0](ipfs-unixfs-exporter-v11.0.0...ipfs-unixfs-exporter-v12.0.0) (2023-02-16)

### ⚠ BREAKING CHANGES

* The options object now accepts preconfigured instances of chunkers and file layouts - these can be imported from this module - see #283 for more

### Features

* accept pre-configured import components as options instead of options for components ([#283](#283)) ([5a38d01](5a38d01))

### Dependencies

* update sibling dependencies ([c59954c](c59954c))
* update sibling dependencies ([b4f6fc8](b4f6fc8))
@github-actions
Copy link

🎉 This PR is included in version ipfs-unixfs-exporter-v12.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant