Skip to content

Latest commit

聽

History

History
134 lines (105 loc) 路 8.11 KB

README.md

File metadata and controls

134 lines (105 loc) 路 8.11 KB
Raw

Launchpad Content

The content package downloads and locally caches content from various common web APIs.

To download content, all you need to do is define content sources and provide credentials as needed.

The following launchpad.json would download jsons and images from the Flickr API to .downloads/flickr-images (a combination of the default .downloads/ directory and the id field of the content source):

{
  "content": {
    "sources": [
      {
        "id": "flickr-images",
        "type": "json",
        "files": {
            "spaceships.json": "https://api.flickr.com/services/feeds/photos_public.gne?format=json&nojsoncallback=1&tags=spaceship",
            "rockets.json": "https://api.flickr.com/services/feeds/photos_public.gne?format=json&nojsoncallback=1&tags=rocket"
        }
      }
    ]
  }
}

See ContentOptions Parameters for a full list of content settings.

Source Settings

Every source needs:

  • id: ID of the individual source (required)
  • type: Source type (default: json)
  • Additional source-specific settings

Currently supported sources are:

Authentication

Some content sources require credentials to access their APIs.

These can all be stored in a .env or .env.local file which will be automatically loaded by launchpad.

.env.local

AIRTABLE_API_KEY=<YOUR_AIRTABLE_API_KEY>

CONTENTFUL_PREVIEW_TOKEN=<YOUR_CONTENTFUL_PREVIEW_TOKEN>
CONTENTFUL_DELIVERY_TOKEN=<YOUR_CONTENTFUL_DELIVERY_TOKEN>
CONTENTFUL_USE_PREVIEW_API=false

SANITY_API_TOKEN=<YOUR_API_TOKEN>

STRAPI_IDENTIFIER=<YOUR_API_USER>
STRAPI_PASSWORD=<YOUR_API_PASS>

launchpad.config.js

export default defineConfig({
	content: {
		sources: [
			{
				id: "airtable-cms",
				type: "airtable",
				apiKey: process.env.AIRTABLE_API_KEY,
			},
			{
				id: "contentful-cms",
				type: "contentful",
				previewToken: process.env.CONTENTFUL_PREVIEW_TOKEN,
				deliveryToken: process.env.CONTENTFUL_DELIVERY_TOKEN,
				usePreviewApi: false,
			},
			{
				id: "sanity-cms",
				type: "sanity",
				apiToken: process.env.SANITY_API_TOKEN,
			},
			{
				id: "strapi-cms",
				type: "strapi",
				identifier: process.env.STRAPI_IDENTIFIER,
			},
		],
	},
});

Post Processing

Once content is downloaded it can be processed to transform text and images. For example, launchpad can convert markdown to html and create scaled derivatives of each image.

ContentOptions

Options for all content and media downloads. Each of these settings can also be configured per ContentSource.

Property Type Default Description
sources Array.<SourceOptions> [] A list of content source options. This defines which content is downloaded from where.
imageTransforms Array.<Object.<string, number>> [] A list of image transforms to apply to a copy of each downloaded image.
contentTransforms Object.<string, string> {} A list of content transforms to apply to all donwloaded content.
downloadPath string '.downloads/' The path at which to store all downloaded files.
credentialsPath string '.credentials.json' The path to the json containing credentials for all content sources.
tempPath boolean '%DOWNLOAD_PATH%/.tmp/' Temp file directory path.
backupPath boolean '%DOWNLOAD_PATH%/.backups/' Temp directory path where all downloaded content will be backed up before removal.
keep boolean '' Which files to keep in dest if clearOldFilesOnSuccess or clearOldFilesOnStart are true. E.g. '\*.json|\*.csv|\*.xml|\*.git\*'
strip string '' Strips this string from all media file paths when saving them locally
backupAndRestore boolean true Back up files before downloading and restore originals for all sources on failure of any single source.
maxConcurrent number 4 Max concurrent downloads.
maxTimeout number 30000 Max request timeout in ms.
clearOldFilesOnSuccess boolean true Remove all existing files in dest dir when downloads succeed. Ignores files that match keep
clearOldFilesOnStart boolean false Will remove all existing files _before_ downloads starts. false will ensure that existing files are only deleted after a download succeeds.
ignoreCache boolean false Will always download files regardless of whether they've been cached
enableIfModifiedSinceCheck boolean true Enables the HTTP if-modified-since check. Disabling this will assume that the local file is the same as the remote file if it already exists.
enableContentLengthCheck boolean true Compares the HTTP header content-length with the local file size. Disabling this will assume that the local file is the same as the remote file if it already exists.
abortOnError boolean true If set to true, errors will cause syncing to abort all remaining tasks immediately
ignoreImageTransformCache boolean false Set to true to always re-generate transformed images, even if cached versions of the original and transformed image already exist.
ignoreImageTransformErrors boolean true Set to false if you want to abort a content source from downloading if any of the image transforms fail. Leaving this to true will allow for non-image files to fail quietly.
forceClearTempFiles boolean true Set to false if you want to keep all contents of the tempPath dir before downloading