Commit Graph

157 Commits

Author SHA1 Message Date
MICHAEL JACKSON dd8df34574 Revert to custom file sending logic 2017-08-26 20:38:31 -07:00
MICHAEL JACKSON 5331120a84 Allow serving dotfiles 2017-08-26 19:50:47 -07:00
MICHAEL JACKSON 30de36aa40 Use req.filename directly 2017-08-26 19:50:45 -07:00
MICHAEL JACKSON e33d49a8cb Log sendFile errors 2017-08-26 19:50:09 -07:00
MICHAEL JACKSON 7d34cc9b88 Add $DEBUG_REDIS env var 2017-08-26 17:44:50 -07:00
MICHAEL JACKSON 50979bfca5 Fix ?module bug with empty filenames 2017-08-26 17:44:29 -07:00
MICHAEL JACKSON e95dcf0028 Do not needlessly redirect 2017-08-24 01:12:26 -06:00
MICHAEL JACKSON 02b0dc91e2 More improvements for ?module 2017-08-24 00:33:58 -06:00
MICHAEL JACKSON 28543a39b6 Add ?module support to server
In "module" mode, the server looks for the "module"/"esnext:main" fields
in package.json for an ES module. Also, it redirects to the real file
path so that relative imports work.
2017-08-23 14:14:21 -07:00
MICHAEL JACKSON 6329a83f67 Use / instead of "index" 2017-08-22 20:16:21 -07:00
MICHAEL JACKSON f64bb6bfd5 Treat URLs with no trailing slashes like files
This is to be consistent with the rest of the API, but also to allow
packages to omit the trailing slash and file name when publishing
multiple builds in the same package.
2017-08-22 18:51:37 -07:00
MICHAEL JACKSON ab3e21b8f8 Be explicit with status codes 2017-08-22 18:44:26 -07:00
MICHAEL JACKSON 7ed062342e Rename logs worker 2017-08-22 17:22:19 -07:00
MICHAEL JACKSON c1c2d1445f Add "module" Cache-Tag to js modules served with ?module 2017-08-22 09:59:19 -07:00
MICHAEL JACKSON 37640b2c8b Cache stats for 1 minute 2017-08-22 09:45:41 -07:00
MICHAEL JACKSON 52367d97a3 Add more spam to the blacklist 2017-08-22 09:22:31 -07:00
MICHAEL JACKSON 47d42addd2 Filter blacklisted packages from stats
Also added better descriptions to stats page.
2017-08-22 09:12:57 -07:00
MICHAEL JACKSON 2a0d32f214 Add /_stats endpoint
Also, remove ingest_stats worker and use the cache instead.
2017-08-22 08:52:30 -07:00
MICHAEL JACKSON c4f3d5bbbc Store error/unique counts per hour/minute 2017-08-21 10:18:10 -07:00
MICHAEL JACKSON be14167718 Use dependencies for unpkg URLs in module mode 2017-08-21 10:17:40 -07:00
MICHAEL JACKSON 066729f17c Re-organize package URL parsing 2017-08-19 11:44:01 -07:00
MICHAEL JACKSON 22c5c550be Move some utils into middleware/utils 2017-08-18 23:09:29 -07:00
MICHAEL JACKSON bc9731abf1 Add experimental ?expand flag
See #24
2017-08-18 17:55:13 -07:00
MICHAEL JACKSON 19d060f753 Add some server tests 2017-08-18 17:33:27 -07:00
MICHAEL JACKSON f87f0962f4 Use text/plain for PATENTS file 2017-08-18 16:47:52 -07:00
MICHAEL JACKSON 6b482f1099 Redirect ?json and /_meta to ?meta 2017-08-18 15:49:12 -07:00
MICHAEL JACKSON e86421240d Redirect requests with invalid query params to cache 2017-08-18 14:50:42 -07:00
MICHAEL JACKSON 7408b24adf Move utilities into middleware/utils 2017-08-18 12:58:48 -07:00
MICHAEL JACKSON 2d03ed9be6 Remove min-download checking 2017-08-18 12:35:45 -07:00
MICHAEL JACKSON 6c6f8814a3 Temporarily disable daily downloads check 2017-08-17 12:23:56 -07:00
MICHAEL JACKSON 504c55a636 Lower minimum daily downloads to 50 2017-08-17 11:55:09 -07:00
MICHAEL JACKSON 8fb2c7810e Improve stats workers
- More accurate expiration times for stats
- Reduce duplication between stats and logs worker
- Expire hostname stats after one week
2017-08-17 11:24:40 -07:00
MICHAEL JACKSON b9c6c0fc61 Add back checkBlacklist middleware 2017-08-16 23:04:59 -07:00
MICHAEL JACKSON 1173f91091 Require packages to be downloaded >= 100x/day
This should make it more difficult for people who are publishing
malicious packages to npm to get them on the CDN.
2017-08-16 23:04:59 -07:00
MICHAEL JACKSON 666d8afc95 Add /_meta endpoint for metadata
Also, add integrity values to metadata.
2017-08-16 17:30:12 -07:00
MICHAEL JACKSON bc609ca825 Combine fetchPackage and findFile into same middleware 2017-08-16 15:58:14 -07:00
MICHAEL JACKSON b6b477e096 Combine parse + blacklist middleware 2017-08-16 15:58:14 -07:00
MICHAEL JACKSON 65c2aa7357 Use function instead of => 2017-08-16 11:58:01 -07:00
MICHAEL JACKSON f542cdafcd Consolidate db connection logic 2017-08-16 10:47:29 -07:00
MICHAEL JACKSON 16cef63d2b Remove query validation
This is now done at the CDN level.
2017-08-16 10:21:05 -07:00
MICHAEL JACKSON 297db18c02 Remove support for v query param 2017-08-15 15:17:48 -07:00
MICHAEL JACKSON 08292b7c87 Cache redirects for 1 minute 2017-08-15 15:17:34 -07:00
MICHAEL JACKSON f5be48f2f4 Add v query param
Fixes #52
2017-08-15 12:37:33 -07:00
MICHAEL JACKSON 9d15462006 Validate NPM package names in URLs 2017-08-15 11:56:08 -07:00
MICHAEL JACKSON db53a296aa Serve .flow files as text/plain 2017-08-12 20:37:29 -07:00
MICHAEL JACKSON e4f5bfe4c0 Ignore symlinks when decompressing packages 2017-08-12 20:31:24 -07:00
MICHAEL JACKSON b64f8e65f9 Try different method for making all files readable 2017-08-12 17:37:54 -07:00
MICHAEL JACKSON db69375e9c Add Cache-Tag headers 2017-08-12 17:23:40 -07:00
MICHAEL JACKSON 5f2805c2e9 Simplify server config 2017-08-12 10:38:50 -07:00
MICHAEL JACKSON 4653ee9e42 Fix failing test 2017-08-12 09:40:53 -07:00
MICHAEL JACKSON 99c1f7098a Better error message for invalid queries 2017-08-12 09:26:06 -07:00
MICHAEL JACKSON c5340f4c53 Use regular functions instead of arrows 2017-08-12 09:18:54 -07:00
MICHAEL JACKSON ac26872341 More consistent log messages 2017-08-12 09:17:35 -07:00
MICHAEL JACKSON cc737f6a9d Consolidate Cloudflare API in single module 2017-08-12 09:14:46 -07:00
MICHAEL JACKSON be4171ffff Tweak import style 2017-08-11 20:27:37 -07:00
MICHAEL JACKSON 182e98bb3a Consolidate url parse/create in PackageURL module 2017-08-11 20:27:27 -07:00
MICHAEL JACKSON 0d004b581b Tweak some cache TTLs 2017-08-11 19:17:02 -07:00
MICHAEL JACKSON 7abb881795 Prefix info-level log messages with "info:" 2017-08-11 07:35:55 -07:00
MICHAEL JACKSON 7e9dfd3292 Remove sentry 2017-08-11 07:29:29 -07:00
MICHAEL JACKSON de18c52eef Add "meta" query param 2017-08-10 23:14:50 -07:00
MICHAEL JACKSON 5d726d6864 Fix error message 2017-08-10 22:58:02 -07:00
MICHAEL JACKSON 3f50bda1dd Remove ResponseUtils 2017-08-10 22:29:44 -07:00
MICHAEL JACKSON 772897e71d Fix tests 2017-08-10 22:23:58 -07:00
MICHAEL JACKSON f54b56b090 Eliminate unnecessary file read 2017-08-10 22:21:13 -07:00
MICHAEL JACKSON a485858381 Prevent multiple concurrent requests for packages
Fixes #38
Fixes #41
2017-08-10 22:11:55 -07:00
MICHAEL JACKSON cb8061f3e1 Prevent multiple concurrent requests for package info 2017-08-10 20:44:40 -07:00
MICHAEL JACKSON 7661950de3 Break middleware out into separate files 2017-08-10 10:12:50 -07:00
MICHAEL JACKSON 8fdc926bb9 Provide default Redis URL 2017-08-10 09:36:19 -07:00
MICHAEL JACKSON 25b8076811 Rename blacklist file 2017-08-10 09:35:55 -07:00
MICHAEL JACKSON 5c4da640c8 Use same URL parsing function for log processing 2017-08-04 21:59:59 -07:00
MICHAEL JACKSON 6b3b279ca3 Add some tests around URL parsing 2017-08-04 21:59:44 -07:00
MICHAEL JACKSON 7d4a4ba495 Block malicious package
See https://a9t9.com/blog/chrome-extension-adware/
Also https://twitter.com/A9T9_com/status/891784248260272128
2017-07-30 18:00:02 -07:00
MICHAEL JACKSON db947a6e9e Small tweak 2017-07-26 21:42:27 -04:00
MICHAEL JACKSON dc1af59fa1 Tweak formatting 2017-07-12 20:03:39 -07:00
Mordy Tikotzky f91bf5abd7 Manually set mime type for TypeScript files 2017-07-12 19:53:02 -07:00
Mordy Tikotzky c68dac5f33 Add tests for extensions with overridden contentTypes 2017-07-12 19:53:02 -07:00
Mordy Tikotzky 3d4f50131a Migrate some files types to use `mime.define` 2017-07-12 19:53:02 -07:00
MICHAEL JACKSON ac09a03c75 Blacklist some malicious packages 2017-07-03 15:29:18 -07:00
MICHAEL JACKSON 3f044cb7e7 Revert "Remove "blacklist" feature"
This reverts commit 24662763b1.
2017-07-03 15:20:45 -07:00
MICHAEL JACKSON bb3e73918a Use OpenRedis 2017-06-21 11:52:32 +10:00
MICHAEL JACKSON 141e440b94 Revert "Fix extraction race condition"
This reverts commit 4aba460a63.
2017-06-21 11:22:35 +10:00
MICHAEL JACKSON 4aba460a63 Fix extraction race condition
This is a fix for a subtle bug that caused unpkg to incorrectly send a
404 response for valid filenames on the initial request for a package.
It also prevents the same package from being downloaded from the
registry by multiple concurrent requests.

Fixes #38
Closes #39
2017-06-18 16:57:26 +10:00
MICHAEL JACKSON 9bc5d47f1e Remove Cache-Control header from HTML content 2017-06-09 13:36:31 -07:00
MICHAEL JACKSON db8f1fa5a0 Simplify middleware 2017-06-09 13:29:21 -07:00
MICHAEL JACKSON c3d9ede460 Return 404 for packages with no version info 2017-06-09 10:10:42 -07:00
MICHAEL JACKSON 40cac70cc2 Remove space from X-Forwarded-For value in logs 2017-06-06 16:00:40 -07:00
MICHAEL JACKSON 84075faeed Simplify server log format 2017-06-06 15:51:25 -07:00
MICHAEL JACKSON 24662763b1 Remove "blacklist" feature 2017-06-06 15:28:08 -07:00
MICHAEL JACKSON 2ac8dc554e Fix Content-Length header in various response helpers 2017-06-01 09:55:58 -05:00
MICHAEL JACKSON ff14e7d28d Encode URIs in redirect headers 2017-06-01 07:48:01 -05:00
MICHAEL JACKSON 08407a1a2d Fix syntax error 2017-06-01 07:22:45 -05:00
MICHAEL JACKSON 77007eacdf More generous gutters and margins on index pages 2017-05-30 10:03:02 -07:00
MICHAEL JACKSON 9dc0630bb3 Return top scores already sorted 2017-05-30 09:20:45 -07:00
MICHAEL JACKSON bca5babb7d Expire all stats keys eventually 2017-05-28 22:41:01 -07:00
MICHAEL JACKSON 1cb6233383 Move workers into server dir 2017-05-26 09:04:03 -07:00
MICHAEL JACKSON d26071ea76 Consolidate some stats server functions 2017-05-25 16:37:54 -07:00
MICHAEL JACKSON c0e8d771d0 Update local tmp dir name 2017-05-25 12:01:29 -07:00
MICHAEL JACKSON 84f297b400 Use SETEX for storing package.json configs
Also, remove option for in-memory cache.
2017-05-25 11:54:15 -07:00
MICHAEL JACKSON f297ba2217 Remove debug statement 2017-05-25 11:39:18 -07:00
MICHAEL JACKSON d2f318f53f Remove unneeded require 2017-05-25 11:31:31 -07:00