Skip to main content

Web API application

List of Starsky Projects​

starsky/starsky docs​

Structure configuration:​

When setup Starksy there are two options to configure the installation. There is a list of required settings. First the appsettings.json is loaded and the environment variables are overwriting features. The command line arguments are shortcuts to set an in-app environment variable.

The order of reading settings​

You could use machine specific configuration files: appsettings.{machinename}.json (and replace {machinename} with your computer name in lowercase)

  1. You can use appsettings.json inside the application folder to set base settings. The order of this files is used to get the values from the appsettings

    • /bin/Debug/net6.0/appsettings.patch.json
    • /bin/Debug/net6.0/appsettings.default.json
    • /bin/Debug/net6.0/appsettings.computername.patch.json
    • /bin/Debug/net6.0/appsettings.json
    • /bin/Debug/net6.0/appsettings.computername.json

  2. Use Environment variables to overwrite those base settings For ThumbnailTempFolder use app__ThumbnailTempFolder (source)
    Dictionaries can be used this way: app__accountRolesByEmailRegisterOverwrite__test@mail.be

  3. Command line arguments in the Cli applications to set in-app environment variables

Required settings to start​

  1. To start it is not mandatory to adjust any settings.

Recommend settings​

  1. ThumbnailTempFolder - For storing thumbnails (default: ./bin/Debug/net6.0/thumbnailTempFolder)
  2. StorageFolder - For the main photo directory (default: ./bin/Debug/net6.0/storageFolder)
  3. DatabaseType - mysql, sqlite or inmemorydatabase are supported (default: sqlite)
  4. DatabaseConnection - The connection-string to the database (default: ./bin/Debug/net6.0/data.db)
  5. CameraTimeZone - The timezone of the Camera, for example Europe/Amsterdam (defaults to your local timezone)

Optional settings​

  1. Structure - The structure that will be used when you import files, has a default fallback.
  2. DependenciesFolder - where store the data of external dependencies used default folder in project
  3. ReadOnlyFolders - Accepts a list of folders that never may be edited, defaults a empty list
  4. AddMemoryCache - Enable caching (default true) The only 2 build-in exceptions are when there are no accounts or you already logged in (default false)
  5. AddSwagger - To show a user interface to show al REST-services (default false)
  6. ExifToolImportXmpCreate - is used to create at import time a xmp file based on the raw image (default false)
  7. AddSwaggerExport - To Export Swagger definitions on startup (default false)
  8. AddLegacyOverwrite- Read Only value for ("Mono.Runtime") (default false)
  9. Verbose - show more console logging (default false)
  10. WebFtp - ftp path, this is used by starskyWebFtpCli
  11. PublishProfiles - settings to configure publish output, used by starskyWebHtmlCli and publish button
  12. ExifToolPath - A path to Exiftool.exe to ignore the included ExifTool
  13. isAccountRegisterOpen - Allow everyone to register an account (default false)
  14. AccountRegisterDefaultRole When a user is new and register an account, give it the role User or Administrator (default User)
  15. ApplicationInsightsConnectionString - Track Telemetry with Microsoft Application Insights (use connection string instead of Instrumentation key) (default disabled)
  16. ApplicationInsightsDatabaseTracking - Track database dependencies (need to have InstrumentationKey) (default disabled)
  17. ApplicationInsightsLog - Add WebLogger output to Application Insights (need to have InstrumentationKey) (default enabled, when key is provided)
  18. useHttpsRedirection - Redirect users to https page. You should enable before going to production. This toggle is always disabled in debug/develop mode (default false)
  19. httpsOn Set all cookies in https Mode. You should enable before going to production. (default false)
  20. Name Name of the application, does not have much effect (default Starsky)
  21. AppSettingsPath To store the settings by user in the AppData folder (default empty string)
  22. UseRealtime Update the user interface realtime default true
  23. UseDiskWatcher Watch the disk for changes and update the database default true
  24. CheckForUpdates Check if there are updates on github and notify the user default true
  25. SyncIgnore Ignore pattern to not include disk items while running sync, uses always unix style and startsWith default list with: /lost+found
  26. ImportIgnore ImportIgnore filter default list with: "lost+found" ".Trashes"
  27. MaxDegreesOfParallelism Number of jobs running in background default 6
  28. MetaThumbnailOnImport Create small thumbnails after import, is very fast default true
  29. EnablePackageTelemetry Telemetry is send for service improvement default true
  30. EnablePackageTelemetryDebug Debug Telemetry default false
  31. AddSwaggerExportExitAfter Quit application after exporting swagger files, should have AddSwagger and AddSwaggerExport enabled default false
  32. NoAccountLocalhost No login needed when on localhost, used in Desktop App
  33. VideoUseLocalTime Use localtime by Camera make and model instead of UTC
  34. SyncOnStartup Sync Database on changes since latest start default true
  35. ThumbnailGenerationIntervalInMinutes Interval to generate thumbnails, to disable use value lower than 3 default 15
  36. GeoFilesSkipDownloadOnStartup Skip download of GeoFiles on startup, recommend to keep this false or null - default false
  37. ExiftoolSkipDownloadOnStartup Skip download of Exiftool on startup, recommend to keep this false or null - default false
  38. AccountRolesByEmailRegisterOverwrite Overwrite the default role for a user by email address, default empty list

Appsettings.json example​

{
  "App": {
    "ThumbnailTempFolder": "Y:\\data\\photodirectory\\temp",
    "StorageFolder": "Y:\\data\\photodirectory\\storage",
    "DatabaseType": "mysql",
    "DatabaseConnection": "Server=mysqlserver.nl;database=dbname;uid=username;pwd=password;",
    "Structure": "/yyyy/MM/yyyy_MM_dd/yyyyMMdd_HHmmss_{filenamebase}.ext",
    "ReadOnlyFolders": ["/2015","/2018"],
    "AddMemoryCache": "true",
    "CameraTimeZone": "America/New_York",
    "ImportIgnore": ["lost+found"]
  }
}

Note: When using a boolean in the json add quotes. Booleans without quotes are ignored

Tip: When using the mysql-setting, make sure the database uses utf8mb4 and as collate utf8mb4_unicode_ci to avoid encoding errors.

Appsettings Notes​

  1. Structure uses slash as directory separators for Linux and Windows
  2. The settings: ExifToolPath, ThumbnailTempFolder and StorageFolder uses the system path directory separators
  3. When using Windows please double escape (\\) system path's

Warmup script​

The default behavior of .NET is to load everything first. To be sure that the application is warm before someone arrives, please check tools/starsky-warmup.sh.

Search Docs​

Advanced queries are supported by the basic search engine.

All text (not number or date) driven search queries use a contain search

Search operators documentation​

Search optionsexampledescription
-tagstestdefault option
-tags-tags="testtag"
-tags-test appleIgnore the keyword test
-tags-Tags-"test"Ignore the keyword test
-tagsapple bananasearch for apple or banana
-title-title="example"
-filepath-filepath="/path"-inurl is the same
-filename-filename="file.jpg"
-filehash-filehash=3DB75N7JJ6FDOPZY4YHGX4TL
-parentdirectory-parentdirectory="/2019"
-description-description="search"
-imageformat-ImageFormat="jpg"include jpeg
-imageformat-ImageFormat="tiff"including dng
-imageformat-ImageFormat="bmp"
-imageformat-ImageFormat="gif"
-imageformat-ImageFormat="png"
-imageformat-ImageFormat="xmp"
-imageformat-ImageFormat="gpx"
-datetime-datetime=1search for yesterday
-datetime-datetime>12 -datetime<2between 2 and 12 days ago
-datetime-datetime=2020-01-01between 00:00:00 and 23:59:59
-datetime-datetime=2020-01-01T14:35:29on this exact time
-addtodatabase-addtodatabase=1search for yesterday
-addtodatabase-addtodatabase>12 -addtodatabase<2between 2 and 12 days ago
-addtodatabase-addtodatabase=2020-01-01between 00:00:00 and 23:59:59
-addtodatabase-addtodatabase=2020-01-01T14:35:29on this exact time
-lastedited-lastedited=1search for yesterday
-lastedited-lastedited>12 -lastedited<2between 2 and 12 days ago
-lastedited-lastedited=2020-01-01between 00:00:00 and 23:59:59
-lastedited-lastedited=2020-01-01T14:35:29on this exact time
-isdirectory-isdirectory=truesearch for folders
-isdirectory-isdirectory=falsesearch for items
-make-make=Applebrand name of the camera
-model-model="iPhone SE"search for camera model
-colorclass-colorclass=1search for colorClass
-colorclass-colorclass=0No Color / None
-colorclass-colorclass=1Purple / Winner
-colorclass-colorclass=2Red / WinnerAlt
-colorclass-colorclass=3Orange / Superior
-colorclass-colorclass=4Yellow / SuperiorAlt
-colorclass-colorclass=5Green / Typical
-colorclass-colorclass=6Azure / TypicalAlt
-colorclass-colorclass=7Blue / Extras
-colorclass-colorclass=8Grey / No name
software-software:"photoshop"Last edited this app

Rest API documentation​

Starsky has a Json restful API. There is a Swagger documentation available at /swagger/index.html and in the documentation there is a API chapter

Tip: Breaking changes are documented in ./history.md

Swagger / OpenAPI​

Swagger is an open-source software framework backed by a large ecosystem of tools that helps developers design, build, document, and consume RESTful Web services. There is an swagger definition. You could enable this by setting the following values:

By default this feature is disabled, please use the AddSwagger definition in the AppSettings or use the following environment variable:

app__AddSwagger=true

This is the default location of the swagger documentation

http://localhost:4000/swagger

Known 'There are critical errors in the following components:'​

When the UI starts there is an Health API check to make sure that some important components works good

Disk Space errors​

  • Storage_StorageFolder There is not enough disk space available on the storage folder location
  • Storage_ThumbnailTempFolder There is not enough disk space available on the thumbnails folder location
  • Storage_TempFolder There is not enough disk space available on the temp folder location

Folder or file not exist errors​

  • Exist_StorageFolder The Storage Folder does not exist, please create it first.
  • Exist_TempFolder The Temp Folder does not exist, please create it first.
  • Exist_ExifToolPath ExifTool is not linked, you need this to write meta data to files.ExifTool. Try to remove the temp folder and run the Application again.
  • Exist_ThumbnailTempFolder The Thumbnail cache Folder does not exist, please create it first.

Date issues​

  • DateAssemblyHealthCheck this setting checks if your current datetime is newer than when this application is build

ApplicationDbContext, Mysql or Sqlite​

There is also a check to make sure the database runs good

Application Insights​

Health issues are also reported to Microsoft Application Insights This only is when a valid key is configured.

Known issues​

DiskWatcher in combination with child folders that have no access​

When using useDiskwatcher: true and there are child folders that are not allowed to read For example the lost+found folder

drwx------ 2 root root  16K Apr 16  2018 lost+found

Then DiskWatcher is stopping and retry 20 times before the state will be disabled

[DiskWatcher] (catch-ed) Access to the path '/mnt/external_disk/lost+found' is denied

Solution: make sure that all child folder are accessible

DiskWatcher in combination with Mac OS APFS Disk​

When you set /System/Volumes/Data to watch for changes this makes the application crash with System.ArgumentOutOfRangeException when a single file is changed. There is currently no solution for this problem other then don't use the Diskwatcher with this location.

Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
   at System.IO.FileSystemWatcher.RunningInstance.ProcessEvents(Int32 numEvents, Byte** eventPaths, FSEventStreamEventFlags* eventFlags, UInt64* eventIds, FileSystemWatcher watcher)
   at System.IO.FileSystemWatcher.RunningInstance.<>c__DisplayClass14_0.<FileSystemEventCallback>b__0(Object o)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.IO.FileSystemWatcher.RunningInstance.FileSystemEventCallback(IntPtr streamRef, IntPtr clientCallBackInfo, IntPtr numEvents, Byte** eventPaths, FSEventStreamEventFlags* eventFlags, UInt64* eventIds)
...