#1572 closed task (fixed)

migrate to new server

Reported by: warner Owned by: zooko
Priority: major Milestone: soon (release n/a)
Component: dev-infrastructure Version: n/a
Keywords: trac website Cc: zooko, davidsarah, secorp
Launchpad Bug:

Description (last modified by daira)

This ticket is to track the task of moving this Trac instance, and the mailing lists, etc, from the old original "org" box (currently in the Undisclosed Location) to the new linode-based "new-org" box.

Remaining Tasks:

  • get basic webserver running
  • move Mailman, list archives, cgi-bin/mailman control panels
  • update MX records to send tahoe-lafs.org mail to new box
  • copy all trac DBs/workdirs to new box
  • activate all trac instances on new box
  • copy all static content (tarballs, deps, debs, all sorts of random stuff)
  • move source trees, canonical darcs repos
  • get post-commit hooks working
  • get any cronjobs migrated
  • move buildmaster, either get buildslave admins to update their configs, or wait for DNS to change
    • add "buildmaster.tahoe-lafs.org" DNS name (pointing at new-org)
    • tell buildslave admins to point buildslaves at that
    • meanwhile, old slaves will just use tahoe-lafs.org and work normally
  • update DNS
  • power down old box, or move to Peter's basement for emergency access
  • set up trac backup-to-git cronjob on new box
  • get SSL cert fixed
  • fix bitcoin-donation box on front page
  • repopulate http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-sdists/
  • check other tahoe-deps directories
  • move the external (non-https) hosted images from the front page to the local server, to hush the mixed-content warning
  • davidsarah noticed that the 'view tickets' link in the nav bar is broken.. it points to https://tahoe-lafs.org/trac/tahoe-lafs//tahoe-lafs.org/trac/tahoe-lafs/wiki/ViewTickets
  • some buildslaves are failing to do darcs fetches. A local 'wget' of the inventory file (on my OS-X 10.6 box) complained about not being able to verify the cert. related?
    • hypothesis is that the old darcs on those boxes is treating HTTPS urls as if they were HTTP, and then failing with the http->https redirect. tcpdump shows one buildslave clearly fetching from port 80 despite being given an HTTPS URL
    • move those buildslaves to Unsupported
    • zooko will mail the owners, ask them to update darcs
    • later, move them back to Supported
  • darcs pushes aren't showing up on the Trac timeline, although the buildbot sees them
    • trac.db and all darcs-repo files need to be gid=source, and all committers must be in the 'source' group
  • bookmarks like http://tahoe-lafs.org/trac/tahoe no longer work (they get redirected to https://tahoe-lafs.org/trac/tahoe , but that gives a "Environment not found" error). This also happens for deeper URLs like http://tahoe-lafs.org/trac/tahoe/ticket/1572 . I think this needs to be rewritten to e.g. https://tahoe-lafs.org/trac/tahoe-lafs
    • still to do: decide whether this should be an internal rewrite or an external redirect. I made it a temporary-redirect for now, need to pick one or the other (if redirect, switch to a permanent-redirect)
  • there's something weird going on with page caching, like the ETag or Date headers are wonky. When I follow a trac link in my browser to a page that I know has just changed (like modifying a wiki page, and the submit button does a POST which returns a redirect back to a GET of the modified page), I see the old contents. Then doing a reload lets me see the new contents.

Change History (27)

comment:1 Changed at 2011-10-26T07:17:25Z by warner

I've started the rsync of /home/source/darcs/tahoe-lafs (1.7GB in 33 trees). The target is /home/warner/incoming

comment:2 Changed at 2011-10-28T07:15:24Z by warner

  • Description modified (diff)

buildbot is migrated

comment:3 Changed at 2011-10-28T07:15:58Z by warner

  • Description modified (diff)

comment:4 Changed at 2011-10-29T00:33:48Z by warner

  • Description modified (diff)

comment:5 Changed at 2011-10-29T00:35:28Z by warner

  • Description modified (diff)

comment:6 Changed at 2011-10-29T00:35:51Z by warner

  • Description modified (diff)

comment:7 Changed at 2011-10-29T01:06:10Z by warner

  • Description modified (diff)

comment:8 Changed at 2011-10-29T16:45:55Z by zooko

  • Description modified (diff)

comment:9 Changed at 2011-10-29T16:49:32Z by warner

  • Description modified (diff)
  • Priority changed from critical to major

comment:10 Changed at 2011-10-29T17:32:27Z by warner

  • Description modified (diff)

comment:11 Changed at 2011-10-29T19:17:24Z by warner

  • Description modified (diff)

comment:12 Changed at 2011-10-29T19:21:24Z by warner

  • Description modified (diff)

comment:13 Changed at 2011-10-29T19:50:14Z by warner

  • Description modified (diff)

comment:14 Changed at 2011-10-29T20:15:43Z by warner

  • Description modified (diff)

comment:15 Changed at 2011-10-29T21:03:02Z by warner

  • Description modified (diff)

comment:16 Changed at 2011-10-29T21:57:02Z by warner

  • Description modified (diff)

comment:17 Changed at 2011-10-31T02:28:38Z by warner

  • Description modified (diff)

Add note about https://tahoe-lafs.org/trac/tahoe/ticket/1572 not working.

Note: having "Tahoe-LAFS" be the full public name of the project is great (since "Tahoe" by itself is hard to search for), and having "tahoe-lafs.org" be the hostname for the project is great (because "tahoe.org" is probably owned by some car company or ski resort). But of course, when talking about it, we use just "tahoe" as an abbreviation. And I don't think it's necessary to make every single directory and URL component spell out the full name. There are a growing number of symlinks and URL rewriting rules to replace "tahoe" with "tahoe-lafs" on the web site, and I think those are going to eventually drive us completely insane. So let's avoid changing anything if we can, identify which URLs have been around the longest, and make them work without introducing new variants.

comment:18 Changed at 2011-10-31T06:39:52Z by warner

  • Description modified (diff)

comment:19 Changed at 2011-10-31T07:03:04Z by warner

Looking more carefully at the HTTP response headers (in Firebug), it looks like every response from the server (including the dynamically-generated ones, from trac) is getting an "Expires" header dated one month in the future. That can't be right. There's also a Cache-Control: max-age=2592000, public.

I'm suspecting that nginx is unconditionally telling browsers to cache the result, when for any uwsgi-based resource, it should not offer caching at all.

I see the same headers on static content (pointing at /chrome), which gets back a correct 304 Not Modified.

comment:20 Changed at 2011-10-31T07:11:43Z by warner

Ok, I think I fixed that problem: there was an overbroad expires 30d; clause in the nginx config, causing "it's ok to cache this for one month" Expires: headers to be added to all pages, including generated ones. I added expires off; clauses to all the dynamic resources, and now things seem to be getting better.

If you've visited any trac or buildbot page in the last few days, you probably have a month-to-go cache entry for it (you might notice the page being shown to you really really fast, like 5ms, when really it should take at least a couple hundred ms to reach the server and generate the page). You'll need to manually reload each of those pages once to clear the effects. Or clear your whole cache. Until you do that, following any link to one of the cached pages will show you the old version.

comment:21 Changed at 2011-11-01T04:44:45Z by warner

  • Description modified (diff)

comment:22 Changed at 2012-03-12T19:05:11Z by davidsarah

  • Keywords trac website added

comment:23 Changed at 2012-03-13T17:59:27Z by zooko

  • Description modified (diff)

comment:24 Changed at 2012-03-13T20:13:21Z by zooko

  • Owner changed from somebody to zooko
  • Status changed from new to assigned

comment:25 Changed at 2012-09-17T23:50:35Z by zooko

  • Description modified (diff)

comment:26 follow-up: Changed at 2013-12-28T13:24:53Z by daira

  • Description modified (diff)

Hmm, not sure why this is still open.

comment:27 in reply to: ↑ 26 Changed at 2013-12-28T13:26:49Z by daira

  • Resolution set to fixed
  • Status changed from assigned to closed

Replying to daira:

Hmm, not sure why this is still open.

Oh, because of this:

bookmarks like ​http://tahoe-lafs.org/trac/tahoe no longer work (they get redirected to https://tahoe-lafs.org/trac/tahoe , but that gives a "Environment not found" error). This also happens for deeper URLs like ​http://tahoe-lafs.org/trac/tahoe/ticket/1572 . I think this needs to be rewritten to e.g. https://tahoe-lafs.org/trac/tahoe-lafs

  • still to do: decide whether this should be an internal rewrite or an external redirect. I made it a temporary-redirect for now, need to pick one or the other (if redirect, switch to a permanent-redirect)

I think this is not important and it's fine as it is.

Note: See TracTickets for help on using tickets.