From 7786ce2a332b0eba4b3ca7c57f906a32e8715da3 Mon Sep 17 00:00:00 2001 From: Bryan Bishop Date: Fri, 29 Oct 2010 19:46:24 -0500 Subject: Starting repo cleanup to make this not so awful --- docs/architecture.mdwn | 1 + docs/architecture/data.mdwn | 19 +++++++++++ docs/architecture/features.mdwn | 7 ++++ docs/architecture/mail.mdwn | 6 ++++ docs/architecture/needed_admin_infrastructure.mdwn | 7 ++++ .../needed_user_facing_infrastructure.mdwn | 37 ++++++++++++++++++++++ docs/architecture/setup_file_overrides.mdwn | 24 ++++++++++++++ 7 files changed, 101 insertions(+) create mode 100644 docs/architecture.mdwn create mode 100644 docs/architecture/data.mdwn create mode 100644 docs/architecture/features.mdwn create mode 100644 docs/architecture/mail.mdwn create mode 100644 docs/architecture/needed_admin_infrastructure.mdwn create mode 100644 docs/architecture/needed_user_facing_infrastructure.mdwn create mode 100644 docs/architecture/setup_file_overrides.mdwn (limited to 'docs') diff --git a/docs/architecture.mdwn b/docs/architecture.mdwn new file mode 100644 index 0000000..581ceb7 --- /dev/null +++ b/docs/architecture.mdwn @@ -0,0 +1 @@ +[[!map pages="architecture/* and ! architecture/*/*"]] diff --git a/docs/architecture/data.mdwn b/docs/architecture/data.mdwn new file mode 100644 index 0000000..a81a268 --- /dev/null +++ b/docs/architecture/data.mdwn @@ -0,0 +1,19 @@ +Data model! + +Dynamic data lookup should opportunisticly cache any associated cheap data. For instance, there are three data from stat(/srv/git/$reponame.git), so looking up one should grab the others. Jules points out that everything here is inherently cached, and I should be using getpw*() for passwd parsing. + +[[!table format=dsv delimiter=# data=""" +datum #stored location #constraint +username #usually $ENV{SUDO_USER} # !~ /^git-|^ikiwiki-/ and =~ /^[a-zA-Z0-9_.][a-zA-Z0-9_.-]+$/ +uid #/etc/passwd # None +email #/etc/passwd GECOS #Email::Valid::Loose->new("-fqdn" => 1, "-fudge" => 0, "-local_rules" => 0, "-mxcheck" => 1, "-tldcheck" => 0 ); +repoaccess #/etc/group git-$reponame entry # None +reponame #/srv/git/$reponame.git # =~ /^[a-z0-9][a-z0-9.-]+$/ +repodescription #/srv/git/$reponame.git/description # =~ /^[\x{0020}-\x{FDCF}\x{FDF0}-\x{FFFD}]{1,80}$/ +repoowner #stat /srv/git/$reponame.git/objects uid; might be better as first non-ikiwiki user in /etc/group git-$reponame entry # None +repoglobalwritable #stat /srv/git/$reponame.git/objects o+w bit # None +repoglobalreadable #stat /srv/git/$reponame.git/objects o+r bit # None +repoikiwikidisable #probably in /srv/git/$reponame.git/config # None +repointernaltemplates #probably in /srv/git/$reponame.git/config # None +repospecialdomain #probably in /srv/git/$reponame.git/config # None +"""]] diff --git a/docs/architecture/features.mdwn b/docs/architecture/features.mdwn new file mode 100644 index 0000000..fbdedb2 --- /dev/null +++ b/docs/architecture/features.mdwn @@ -0,0 +1,7 @@ +An in-distributed-repo model for + +- code +- blogs +- bugs +- web pages +- [[mail]] diff --git a/docs/architecture/mail.mdwn b/docs/architecture/mail.mdwn new file mode 100644 index 0000000..61bbfcd --- /dev/null +++ b/docs/architecture/mail.mdwn @@ -0,0 +1,6 @@ +I can presumably get mailing lists into ikiwiki using one of the following: + +- MLM+Ikiwiki [comment-by-mail plugin](http://ikiwiki.info/todo/comment_by_mail/) +- MLM+Ikiwiki [mailbox plugin](http://pivot.cs.unb.ca/git/?p=ikimailbox.git;a=summary) + +But how would we get it into git? Might we have to do that externally? diff --git a/docs/architecture/needed_admin_infrastructure.mdwn b/docs/architecture/needed_admin_infrastructure.mdwn new file mode 100644 index 0000000..59365c2 --- /dev/null +++ b/docs/architecture/needed_admin_infrastructure.mdwn @@ -0,0 +1,7 @@ +- DONE: HTTP AUTH from PAM through the magic of mod-auth-pam +- DONE: HTTP access from /etc/group through the magic of mod-auth-sys-group +- DONE: Shell security through rbash and privilege escalation security (for mkwiki, mkuser, etc) through sudo +- DONE: Guaranteed namespace coherence with the vhosting of cgi, secure, and normal with the aid of a wildcard SSL certificate +- TODO: Work out how to pass mailing list emails around. Probably either ssh or vserver namespace magic. +- DONE: manual [[setup file overrides]], /etc/ikiwiki/piny/$reponame.setup.pl, included directly with 'do' before dumping state. +- DONE: rebuildrepo diff --git a/docs/architecture/needed_user_facing_infrastructure.mdwn b/docs/architecture/needed_user_facing_infrastructure.mdwn new file mode 100644 index 0000000..1e7284a --- /dev/null +++ b/docs/architecture/needed_user_facing_infrastructure.mdwn @@ -0,0 +1,37 @@ +TODO: user-facing documentation on piny-web +TODO: help system on pinyshell (display on login) + +It'd be nice to maintain feature parity between command line access and web access. + +[[!table data=""" +feature |command line |web +anonymous account creation |DONE: ssh createuser@piny.be |TODO: CGI frontend to adduser +password modification |DONE: /srv/rbin/passwd |TODO: Authen::PAM chauthtok CGI jblake: the pam interface was *really* not designed with http in mind though if you hard-code the interaction script it's not hard at all +repo creation, deletion |DONE: /srv/rbin/newrepo, rmrepo, lsrepo |TODO: CGI frontend +repo user management |DONE: /srv/rbin/addaccess, rmaccess, lsaccess |TODO: CGI frontend +[[mailing list addition|mail]] |TODO: /srv/rbin/addlist, dependent on getting sympa up |TODO: CGI frontend +authorized_keys modification |DONE: /srv/rbin/readkeys, writekeys, appendkeys |TODO: CGI frontend, though i am not convinced anyone would care +disable password auth if ssh key|TODO |Ha ha NEVER +commit access |DONE: git+ssh://piny.be/srv/git/whatever.git |DONE: Ikiwiki+mod-auth-sys-group/mod-auth-pam, though I am told there is a magic hook to allow public editing through git:// +wysiwyg editing tool |Ha ha no |TODO: wmd a good option, though needs UI tweaks +fork |TODO: /srv/rbin/fork |TODO: CGI frontend +upload and commit container contents |no |TODO: CGI frontend +user or repo activity data aggregation |TODO: /srv/rbin/activity |TODO: rss generation, possibly install a dynamic rss reader +help system |lock down 'man', print introduction on login |command documentation should be rendered online, perhaps +config tweaking |PARTIAL: pinyconfig |TODO: CGI frontend +"""]] + +Needed config tweakables + +- TODO: ikiwiki-editable vs. git-only +- TODO: toggle ikiwiki autoindexing +- TODO: optional use of repository subdirectory as srcdir +- TODO: globally readable repos vs. group readable repos +- TODO: globally writable repos vs. group writable repos +- TODO: editable ikiwiki templates vs. default layout +- TODO: user-configured domain vs. default piny domain + +Other + +- TODO: overall site layout/theme/css +- TODO: front page to piny server editable by sysadmin ("here's what's on this server; readme; faq; about;") diff --git a/docs/architecture/setup_file_overrides.mdwn b/docs/architecture/setup_file_overrides.mdwn new file mode 100644 index 0000000..6133a22 --- /dev/null +++ b/docs/architecture/setup_file_overrides.mdwn @@ -0,0 +1,24 @@ +Use cases: + +### Adding post-update hooks, which requires diverting Ikiwiki's hook. + + mv /srv/git/poop.git/hooks/post-update /srv/git/poop.git/hooks/post-update-ikiwiki + cat > /srv/git/poop.git/hooks/post-update + #!/bin/sh + /srv/git/poop.git/hooks/post-update-ikiwiki & + echo This is a hook that does things! + cat > /srv/ikiwiki/piny/poop.setup.pl + foreach(@{$conf->{wrappers}}) { $_->{wrapper} =~ s/post-update/post-update-ikiwiki/; }; + rebuildrepo poop + +### Adding or removing plugins, or plugin confuration variables. + + cat > /srv/ikiwiki/piny/poop.setup.pl + push(@{$conf->{add_plugins}}, 'txt'); + rebuildrepo poop + +### Any other special confuration requests from hosted projects. + + cat > /srv/ikiwiki/piny/poop.setup.pl + $conf->{teximg_prefix} .= "\n\\newcommand{\\unit}[1]{\\ensuremath{\\, \\mathrm{#1}}}" + rebuildrepo poop -- cgit v1.2.3 From e6738da715d990092fd11258d1f0a1520cb45ff2 Mon Sep 17 00:00:00 2001 From: Bryan Bishop Date: Fri, 29 Oct 2010 19:50:51 -0500 Subject: Moved issues/ into docs/issues/ --- docs/issues.mdwn | 6 ++++ ...t_now_supports_per_directory_configuration.mdwn | 17 +++++++++ docs/issues/debian_packaging_concerns.mdwn | 42 ++++++++++++++++++++++ docs/issues/not_tracking_templates.mdwn | 9 +++++ .../plain_git_repository_hosting_needed.mdwn | 10 ++++++ docs/issues/shared_underlay.mdwn | 13 +++++++ docs/issues/wmd.mdwn | 12 +++++++ issues.mdwn | 6 ---- ...t_now_supports_per_directory_configuration.mdwn | 17 --------- issues/debian_packaging_concerns.mdwn | 42 ---------------------- issues/not_tracking_templates.mdwn | 9 ----- issues/plain_git_repository_hosting_needed.mdwn | 10 ------ issues/shared_underlay.mdwn | 13 ------- issues/wmd.mdwn | 12 ------- 14 files changed, 109 insertions(+), 109 deletions(-) create mode 100644 docs/issues.mdwn create mode 100644 docs/issues/cgit_now_supports_per_directory_configuration.mdwn create mode 100644 docs/issues/debian_packaging_concerns.mdwn create mode 100644 docs/issues/not_tracking_templates.mdwn create mode 100644 docs/issues/plain_git_repository_hosting_needed.mdwn create mode 100644 docs/issues/shared_underlay.mdwn create mode 100644 docs/issues/wmd.mdwn delete mode 100644 issues.mdwn delete mode 100644 issues/cgit_now_supports_per_directory_configuration.mdwn delete mode 100644 issues/debian_packaging_concerns.mdwn delete mode 100644 issues/not_tracking_templates.mdwn delete mode 100644 issues/plain_git_repository_hosting_needed.mdwn delete mode 100644 issues/shared_underlay.mdwn delete mode 100644 issues/wmd.mdwn (limited to 'docs') diff --git a/docs/issues.mdwn b/docs/issues.mdwn new file mode 100644 index 0000000..8942268 --- /dev/null +++ b/docs/issues.mdwn @@ -0,0 +1,6 @@ +List of open issues; see [[templates/issues]] + +[[!inline pages="issues/* and !issues/*/* and !issues/done and !link(issues/done) and !link(tag/closed)" show=0 rootpage="issues" postformtext="Add a new issue titled:" actions="yes"]] + +For issues, use the following template: [[!edittemplate template="templates/issues" match="issues/*"]] + diff --git a/docs/issues/cgit_now_supports_per_directory_configuration.mdwn b/docs/issues/cgit_now_supports_per_directory_configuration.mdwn new file mode 100644 index 0000000..380981a --- /dev/null +++ b/docs/issues/cgit_now_supports_per_directory_configuration.mdwn @@ -0,0 +1,17 @@ +* Status: [[!taglink closed]] +* Assigned to: [[!taglink jrayhawk]] +* Priority: [[!taglink now]] +* Opened by: jrayhawk + +### Discussion + +On Tue, Oct 06, 2009 at 11:40:11AM +0200, Lars Hjemli wrote: +> If you want auto-discovery of repositories similar to gitweb, the +> 'scan-path' option will automatically include $GITDIR/cgitrc for each +> repository. + +do this instead of the current goofy cgitrc.d thing we got going now. + +### jrayhawk 20101029 + +We've moved over to just inferring paths and using the gitweb.owner variable, now. diff --git a/docs/issues/debian_packaging_concerns.mdwn b/docs/issues/debian_packaging_concerns.mdwn new file mode 100644 index 0000000..aca7000 --- /dev/null +++ b/docs/issues/debian_packaging_concerns.mdwn @@ -0,0 +1,42 @@ +* Status: [[!taglink open]] +* Assigned to: [[!taglink jrayhawk]] +* Priority: [[!taglink now]] +* Opened by: jrayhawk + +### Discussion + +There are a variety of things that wind up needing to be manually handled outside of the package installation in order to get piny working. It would be nice to correct all these. + +* /etc/piny.conf +* /etc/sudoers.d/pinyadmin +* /etc/sv/git-daemon/run + * --base-path=/var/cache /var/cache/git is not a very good place at all; what do we do about that? +* /srv/rbin +* mkdir /srv/git +* mkdir /etc/piny +* mkdir /etc/apache2/piny-available +* mkdir /etc/apache2/piny-enabled +* mkdir /srv/www/$ikiwiki_destdir/repos +* mkdir /etc/ikiwiki/wikilist.d +* lots of /etc/cgitrc changes +* mkdir /srv/www/$ikiwiki_destdir/static +* ln /usr/share/cgit/cgit.css /srv/www/$ikiwiki_destdir/static +* ln /usr/share/cgit/logo.png /srv/www/$ikiwiki_destdir/static +* /srv/git-template.git +* change GROUP to shadow in /etc/apache2/envvars + +* dependencies + * cgit + * libyaml-tiny-perl + * gcc + * libc6-dev + * dvipng + * texlive-science + * graphviz + * perlmagick + * libcgi-formbuilder-perl + * git-daemon-run + * apache2 + * libapache2-mod-auth-sys-group + * libapache2-mod-auth-pam + diff --git a/docs/issues/not_tracking_templates.mdwn b/docs/issues/not_tracking_templates.mdwn new file mode 100644 index 0000000..f745e2f --- /dev/null +++ b/docs/issues/not_tracking_templates.mdwn @@ -0,0 +1,9 @@ +* Status: [[!taglink closed]] +* Assigned to: [[!taglink jrayhawk]] +* Priority: [[!taglink now]] +* Opened by: jrayhawk + +### Discussion + +A bunch of the templates need to be removed; Joey added an 'add_templates' option that allows us to do nested fallback. + diff --git a/docs/issues/plain_git_repository_hosting_needed.mdwn b/docs/issues/plain_git_repository_hosting_needed.mdwn new file mode 100644 index 0000000..8bcb64b --- /dev/null +++ b/docs/issues/plain_git_repository_hosting_needed.mdwn @@ -0,0 +1,10 @@ +* Status: [[!taglink open]] +* Assigned to: [[!taglink jblake]] +* Priority: [[!taglink now]] +* Opened by: jrayhawk + +### Discussion + +We really badly need to be able to host simple non-ikiwiki git repos; it'd probably be best to do so without invoking the ikiwiki engine. + +This should be doable through both newrepo and pinyconfig diff --git a/docs/issues/shared_underlay.mdwn b/docs/issues/shared_underlay.mdwn new file mode 100644 index 0000000..82901c7 --- /dev/null +++ b/docs/issues/shared_underlay.mdwn @@ -0,0 +1,13 @@ +* Status: [[!taglink open]] +* Assigned to: [[!taglink jrayhawk]] +* Priority: [[!taglink now]] +* Opened by: jrayhawk + +### Discussion + +The 'piny-shared' unified underlay repository causes problems with things like the FormattingHelp link in the editpage. These are difficult to correct. + +Solutions: + +* Deunify the underlay repository, then use vserver piny hashify to unify storage. Not sure this is a good idea if I want to migrate to LXC. +* Get Joey or Josh to make provisions for my usage model. diff --git a/docs/issues/wmd.mdwn b/docs/issues/wmd.mdwn new file mode 100644 index 0000000..f126b80 --- /dev/null +++ b/docs/issues/wmd.mdwn @@ -0,0 +1,12 @@ +* Status: [[!taglink open]] +* Assigned to: [[!taglink jrayhawk]] +* Priority: [[!taglink soon]] +* Opened by: jrayhawk + +### Discussion + +Get wmd working + +Also get wmd splitscreen working + +It would be nice if something freely redistributable came along. diff --git a/issues.mdwn b/issues.mdwn deleted file mode 100644 index 8942268..0000000 --- a/issues.mdwn +++ /dev/null @@ -1,6 +0,0 @@ -List of open issues; see [[templates/issues]] - -[[!inline pages="issues/* and !issues/*/* and !issues/done and !link(issues/done) and !link(tag/closed)" show=0 rootpage="issues" postformtext="Add a new issue titled:" actions="yes"]] - -For issues, use the following template: [[!edittemplate template="templates/issues" match="issues/*"]] - diff --git a/issues/cgit_now_supports_per_directory_configuration.mdwn b/issues/cgit_now_supports_per_directory_configuration.mdwn deleted file mode 100644 index 380981a..0000000 --- a/issues/cgit_now_supports_per_directory_configuration.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -* Status: [[!taglink closed]] -* Assigned to: [[!taglink jrayhawk]] -* Priority: [[!taglink now]] -* Opened by: jrayhawk - -### Discussion - -On Tue, Oct 06, 2009 at 11:40:11AM +0200, Lars Hjemli wrote: -> If you want auto-discovery of repositories similar to gitweb, the -> 'scan-path' option will automatically include $GITDIR/cgitrc for each -> repository. - -do this instead of the current goofy cgitrc.d thing we got going now. - -### jrayhawk 20101029 - -We've moved over to just inferring paths and using the gitweb.owner variable, now. diff --git a/issues/debian_packaging_concerns.mdwn b/issues/debian_packaging_concerns.mdwn deleted file mode 100644 index aca7000..0000000 --- a/issues/debian_packaging_concerns.mdwn +++ /dev/null @@ -1,42 +0,0 @@ -* Status: [[!taglink open]] -* Assigned to: [[!taglink jrayhawk]] -* Priority: [[!taglink now]] -* Opened by: jrayhawk - -### Discussion - -There are a variety of things that wind up needing to be manually handled outside of the package installation in order to get piny working. It would be nice to correct all these. - -* /etc/piny.conf -* /etc/sudoers.d/pinyadmin -* /etc/sv/git-daemon/run - * --base-path=/var/cache /var/cache/git is not a very good place at all; what do we do about that? -* /srv/rbin -* mkdir /srv/git -* mkdir /etc/piny -* mkdir /etc/apache2/piny-available -* mkdir /etc/apache2/piny-enabled -* mkdir /srv/www/$ikiwiki_destdir/repos -* mkdir /etc/ikiwiki/wikilist.d -* lots of /etc/cgitrc changes -* mkdir /srv/www/$ikiwiki_destdir/static -* ln /usr/share/cgit/cgit.css /srv/www/$ikiwiki_destdir/static -* ln /usr/share/cgit/logo.png /srv/www/$ikiwiki_destdir/static -* /srv/git-template.git -* change GROUP to shadow in /etc/apache2/envvars - -* dependencies - * cgit - * libyaml-tiny-perl - * gcc - * libc6-dev - * dvipng - * texlive-science - * graphviz - * perlmagick - * libcgi-formbuilder-perl - * git-daemon-run - * apache2 - * libapache2-mod-auth-sys-group - * libapache2-mod-auth-pam - diff --git a/issues/not_tracking_templates.mdwn b/issues/not_tracking_templates.mdwn deleted file mode 100644 index f745e2f..0000000 --- a/issues/not_tracking_templates.mdwn +++ /dev/null @@ -1,9 +0,0 @@ -* Status: [[!taglink closed]] -* Assigned to: [[!taglink jrayhawk]] -* Priority: [[!taglink now]] -* Opened by: jrayhawk - -### Discussion - -A bunch of the templates need to be removed; Joey added an 'add_templates' option that allows us to do nested fallback. - diff --git a/issues/plain_git_repository_hosting_needed.mdwn b/issues/plain_git_repository_hosting_needed.mdwn deleted file mode 100644 index 8bcb64b..0000000 --- a/issues/plain_git_repository_hosting_needed.mdwn +++ /dev/null @@ -1,10 +0,0 @@ -* Status: [[!taglink open]] -* Assigned to: [[!taglink jblake]] -* Priority: [[!taglink now]] -* Opened by: jrayhawk - -### Discussion - -We really badly need to be able to host simple non-ikiwiki git repos; it'd probably be best to do so without invoking the ikiwiki engine. - -This should be doable through both newrepo and pinyconfig diff --git a/issues/shared_underlay.mdwn b/issues/shared_underlay.mdwn deleted file mode 100644 index 82901c7..0000000 --- a/issues/shared_underlay.mdwn +++ /dev/null @@ -1,13 +0,0 @@ -* Status: [[!taglink open]] -* Assigned to: [[!taglink jrayhawk]] -* Priority: [[!taglink now]] -* Opened by: jrayhawk - -### Discussion - -The 'piny-shared' unified underlay repository causes problems with things like the FormattingHelp link in the editpage. These are difficult to correct. - -Solutions: - -* Deunify the underlay repository, then use vserver piny hashify to unify storage. Not sure this is a good idea if I want to migrate to LXC. -* Get Joey or Josh to make provisions for my usage model. diff --git a/issues/wmd.mdwn b/issues/wmd.mdwn deleted file mode 100644 index f126b80..0000000 --- a/issues/wmd.mdwn +++ /dev/null @@ -1,12 +0,0 @@ -* Status: [[!taglink open]] -* Assigned to: [[!taglink jrayhawk]] -* Priority: [[!taglink soon]] -* Opened by: jrayhawk - -### Discussion - -Get wmd working - -Also get wmd splitscreen working - -It would be nice if something freely redistributable came along. -- cgit v1.2.3 From 62f811460f400b73271b43d1d91c5c65f09c1975 Mon Sep 17 00:00:00 2001 From: Bryan Bishop Date: Fri, 29 Oct 2010 19:51:55 -0500 Subject: fixed the issues.mdwn index display (i think) --- docs/issues.mdwn | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/issues.mdwn b/docs/issues.mdwn index 8942268..68be754 100644 --- a/docs/issues.mdwn +++ b/docs/issues.mdwn @@ -1,6 +1,6 @@ List of open issues; see [[templates/issues]] -[[!inline pages="issues/* and !issues/*/* and !issues/done and !link(issues/done) and !link(tag/closed)" show=0 rootpage="issues" postformtext="Add a new issue titled:" actions="yes"]] +[[!inline pages="docs/issues/* and !docs/issues/*/* and !docs/issues/done and !link(docs/issues/done) and !link(tag/closed)" show=0 rootpage="issues" postformtext="Add a new issue titled:" actions="yes"]] -For issues, use the following template: [[!edittemplate template="templates/issues" match="issues/*"]] +For issues, use the following template: [[!edittemplate template="templates/issues" match="docs/issues/*"]] -- cgit v1.2.3