Question thread #3

[pauamma]

It's time for another question thread!

The rules:

- You may ask any dev-related question you have in a comment. (It doesn't even need to be about Dreamwidth, although if it involves a language/library/framework/database Dreamwidth doesn't use, you will probably get answers pointing that out and suggesting a better place to ask.)
- You may also answer any question, using the guidelines given in To Answer, Or Not To Answer.

Comments

Testing

[randomling]

Hi!

So I'm working on Bug 2886, and I've reached the point (with lots of help!) where I MIGHT have a working patch. And now I need to test it.

And I'm kind of stuck, to be honest.

How do I go about designing a testing plan for a bug like this? What kind of use cases do I need to take into account?

Thanks, guys!

Re: Testing

[snakeling]

Well, you've got 3 fields for each link, so it's a matter of testing how it works depending on what's filled:

- all 3 fields
- text + URL
- title + URL
- text + title
- text only
- title only
- URL only

Also, what happens if you put a URL in the title field? Text in the URL field? What happens if people use the URL to a picture? HTML in the title or text field?

Remember, when testing, you need to test both what you implemented, and what already existed, to make sure you didn't break it ;)

Good luck!

Re: Testing

[pauamma]

A few thoughts off the top of my head, everyone feel free to expand, elaborate, add test cases...
- Links without a title (should still be possible to specify them, and they should still display correctly)
- Links with an empty title
- Links with a not-empty title that's shorter than the maximum allowed
- Links with a not-empty title that's exactly the maximum allowed
- Links with a not-empty title that's 1 byte longer than the maximum allowed
- Links with a not-empty title longer than that
- Links with a not-empty title with a Unicode character that straddles the maximum allowed (eg, if the maximum allowed is 40 bytes, what happens if a 2-byte Unicode character takes positions 40 and 41?)
- Links with HTML markup characters in the title (eg, innocuous-looking text"></a><script src="http://haxxor.example.net/pwned.js" /><a title=")
- Links with entities in the title, but no markup (like I &hearts; titles)
- Links with a single or double quote in the title

Finally, it shouldn't break any existing automated tests (which isn't to say that all tests in our test suite need to pass, but any test that passed should still pass, and any test that failed should fail in roughly the same way). Note that the test suite can be overwhelming, so you may want to leave that for a reviewer/committer to check.

[snakeling]

Still working on my script to post Pinboard bookmarks to Dreamwidth. (And much thanks to both you and foxfirefey for your answers last time. I completely forgot to get back at you, my bad entirely. I'll try to be better this time :) )

I'm stuck on, I think, a fairly easy question, that I nevertheless can't find an answer to.

To use the Dreamwidth API (and the Pinboard one), I need the user's password. My script is PHP, and I tell people installing it to do so outside of the public folders, to reduce problems, but even though, I'm not particularly happy at having to store the password, not even encrypted, in the script. Everything I can find about storing passwords securely is about sites that ask you to login, but I do the opposite: I send the password outside. (Unless I'm a dunce and there is a totally different solution I didn't think of.)

I need to store the script on a web server, because the point is for it to run automatically without human input.

Anyone has the beginning of a solution for me?

[pauamma]

Ideally, the directory the password (and script) is in should be outside the Apache document tree (which may be what you mean by "outside the public folders). If that's not possible, you might be able to handle it without a gaping hole by having the password file in a separate directory that also contains a .htaccess that disables directory indexing and that denies HTTP access to all files by requiring authentication and giving an empty list of authentication methods, Thus, the password data will only be readable by locally-running scripts and won't leak out if someone tries to wget the password file by its URL. (This fails if an attacker manages to get their own code running on the server(*), but there's little you can do to protect from that no matter which approach you use.)

(*) If your script lives on a shared server, which is common for PHP, you should make sure that the server operator enforces separation between customers, eg by having a separate Unix account for each virtual domain's directory tree or something along those lines.

[snakeling]

outside the Apache document tree (which may be what you mean by "outside the public folders)
That's indeed what I meant.

you should make sure that the server operator enforces separation between customers
Mine does, and so does the host I recommend, but I'll have to add this to the requirements, then.

Thanks!

[denise]

also, I should add, we have plans to implement OAuth, which will let you authorize an external service to have access to your account, etc. Someday!

Would it work to store the login cookies, instead of the passwords?

[snakeling]

Would it work to store the login cookies, instead of the passwords?
I have no idea! XD I'm using the script as a learning process actually, and I'm making it up as I go along, probably reinventing the wheel a couple of times.

I'm going to keep your suggestion in mind for when I have time to tackle the documentation. Thanks!

[exor674]

If you can bribe someone to help me with the user facing bits... >_> ( also uh, there should totally be a bug for OAuth stuffsies )

[chagrined]

I am only newly poking at the DW code and am new to most of this. I don't have enough understanding yet to ask interesting questions. But here is an uninteresting question I have about something!

In cgi-bin/bml/scheme/global.look, this is line 171:
GRIN=>{S}&lt;grin&gt;

Does that actually serve a purpose somewhere? It seems like all <?GRIN?> would do if used in a BML document is cause <grin> to print, which then wouldn't show up to the user, but wouldn't do anything else either. Is this just a silly programming easter egg of some kind, or does this actually do something I'm not seeing? (I also did a rgrep and couldn't find <?GRIN?> occurring anywhere.) eta: Oh, wait. I'm realizing now it'd make &lt;grin&gt; print, not <grin>. So the user would indeed see <grin>, I guess?

[pauamma]

Yes, they would indeed see <grin>. Also, the .look files for the legacy (pre-TT) skins have that unpleasant wrinkle that they define in all-uppercase stuff that's then referenced in all-lowercase elsewhere. (I got burned by that one when trying to clean them up early on in DW's life, IIRC.) In this case, though, it lr ooks like it's really not used, UPPERCASE, lowercase, or mixEd CAsE. (And it was already present in the initial version of that file Brad checked in on LJ, so it's really old.

[chagrined]

Hum. Well if it's super old maybe long ago in LJ code it did serve a purpose, but now doesn't? Like idk maybe Brad had <grin> print out on some pages? That seems kind of silly. ;p

Also, it is very useful to know that about the case! I think I first searched for it in all-uppercase and then did a search with that "-i" non-case-sensitive option also just to be sure. :)

[chagrined]

Another question! I am poking at MoodThemeChooser.pm (and other things) in an attempt to change out places where tables were used for layout purposes, to use CSS positioning instead. In MoodThemeChooser.pm there is this variable $moodtheme_extra which takes its value somehow from the hook mood_theme_extra_content? And then another variable $show_special which is set based on that? (I am still just trying to figure out Perl, lol.) ANYWAY. I cannot figure out where mood_theme_extra_content comes from or what it is supposed to signify. I read the wiki page on hooks and the old LJ documentation on them but I'm still confused. And I did an rgrep search but couldn't find mood_theme_extra_content anywhere else. It's making me scratch my head, b/c I can't figure out how I should change the CSS version of the layout in relation to this element.

As far as I can tell, with the table, what it changes is that if $moodtheme_extra is false, then all the example "mood theme" images/descriptions (which are contained in <td> elements) are in ONE row. If $moodtheme_extra is true, then after every 2nd image it inserts a new <tr>. But not knowing what the heck mood_theme_extra_content is, I don't know why it's doing that or whether I should also do that. I wasn't sure also if maybe this is related to the width of the table, b/c I just told my CSS positioning to size the width of that element automatically, but ...if mood_theme_extra_content is related to mood themes that are extra super wide or something, maybe that is why it happened? (I'm positioning those elements with float:left so I could just add in a clear:left probably to the same effect.) (eta: or set a maximum width to the containing element to eliminate the need for this workaround?)

Anyway, confusion. I figure also for testing purposes it's important to know what that hook means so I can test the two different cases. On that end, also, I guess I have another question. How do I make a user account in my dreamhack a "paid account" so I can test paid features? Maybe there is an obvious answer to this somewhere but I couldn't find it on the wiki.

[denise]

For the paid account: go to /admin/priv while logged into the system account or someone with admin:*, give yourself the 'payments' priv, then go to /admin/pay and enter the username of the account you want to upgrade. the subsequent bits should be self-explanatory *G*

meanwhile, without looking into it too closely, i'm going to guess that mood_theme_extra_content is almost certainly some ljcom hook (the non-free lj code), and therefore we don't have it -- most (though not all) of the hooks defined in the code are. To look for where in the code it's referenced, you'd do:

rgrep -i mood_theme_extra_content $LJHOME/cvs/dw-free/* | more
rgrep -i mood_theme_extra_content $LJHOME/cvs/dw-nonfree/* | more

Since neither returns any results except the bit in the widget, it's pretty certain that it was a ljcom-specific hook (and now i'm curious; i looked it up (don't worry, you aren't supposed to know where to find it, I just happened to) and it looks like it has to do with sponsored mood themes, which we don't have. (There are a lot of hooks scattered around the code that LJ used to pull in ljcom-specific things like that -- some of them we've repurposed for other dw-nonfree things that are useful, some of them we just ignore.) So you can safely ignore it.

[chagrined]

Yeah, I did
    rgrep -i mood_theme_extra_content * > whatevertempfilename
while in the parent directory of my dreamhack, and those were the only places where it showed up.

What categorizes "ljcom" things? Are they all related to sponsored things, or are they code bits that are not open source or something? I wasn't really positive why the Mercurial repositories have both the "dw-free" and "dw-nonfree" branches (but the live directory doesn't?).

Since DW has no sponsored mood themes, in working on editing that module to use CSS positioning should I also edit it to not take that hook+corresponding variables into account? Rather than having lines of code that serve no purpose / do nothing? Or comment them out or something?

Also there were some hooks I was unfamiliar with also when I was poking at S2PropGroup.pm (which I've taken a break from to work on others instead). I don't remember them off the top of my head, but, it looks like that thing you linked to might be a good place to look up some of them. If there are some that don't appear to be registered anywhere in the DW build that I have, but I find them on that site, is it usually the case that they are ljcom things that are not in use? Should I double check any time I find one like that before assuming that it doesn't serve a purpose?

P.S. Thanks, it was causing me immense aggravation not being able to find that thing anywhere, ha.

[chagrined]

I'm baaaaaaaaaaaack?

Thanks to denise's response to my question I was able to figure out the hooks in MoodThemeChooser.pm and now I think I have a version that correctly uses CSS positioning. Only then I tried to figure out how to generate a patch and got totally lost. I tried to follow the instructions on the wiki for maintenance/syncing my CVS installation/whatever first. Okay. But from what I can gather ...the instructions on patching on most of those pages assume I was doing my edits in the CVS directory and generating patches as I went along? Now I'm just lost and frustrated. One of the "getting started" pages on the wiki said that you could do your editing in either the live directory or in the CVS directories and that it had something to do with personal preference for workflow? So I did it in the live directory but now can't figure out how to get THAT to sync with my CVS directory. The problem is that I did change files other than MoodThemeChooser.pm, so when I generated a "compare everything" patch it had a lot of other changes I don't want to include in this patch. But I don't want to JUST declare MoodThemeChooser.pm b/c I also changed the files governing the CSS rules for that page (in both htdocs and ssldocs). So I guess I could generate three separate patches, one for each of those files, but that seems annoying, and then if I had more than three files could get old really soon.

I backed up the other files in my live directory that I changed anyway so that's why I wanted to just sync/override the whole thing, then I could copy my modified MoodThemeChooser.pm & related files back in there to only generate the patch based on those. Also though when I did the patch comparing everything in my live directory with the CVS one, there was a lot of weird stuff in there I definitely didn't change, like, config files related to ...time and things like that? (I don't have the patch file in front of me right now).

Uh basically I'm just confused and frustrated and wondering whether it's best to wipe my entire installation, reset it from scratch and then try to figure out this other method of doing my edits in the CVS directory instead? (And then I use patches to make those edits apply to the live directory so I can see them when I go to the site or something? Is that how that works?)

[denise]

Fear not, you don't have to delete everything! It is possible to sync between live and cvs fairly easily. These instructions will walk you through setting up an alias to do it so you don't have to run the whole command every time.

(Also, if you wind up with changes in a patch that you don't want (other changed files, etc) you can just delete those changes. Make sure you get everything from the part that starts "diff -r" to the beginning of the next "diff -r", or else there will be problems applying the patch).

1) First, do this: http://wiki.dwscoalition.org/notes/Dreamhack_getting_started#Protect_your_configs_from_updates (that will prevent your config changes from being included in every patch ever).

2) Second, while logged into your DH account, cd ~/bin and pico -w sc. In the window that results, paste:
#!/bin/bash $LJHOME/bin/cvsreport.pl -s -c

Hit ctrl-X, say yes when it asks you to save, and once you're back at the command line type chmod +x sc. Then, pico -w sl and paste:
#!/bin/bash $LJHOME/bin/cvsreport.pl -s -l

Ctrl-X, save, and type chmod +x sl.

Once you've done that, you'll be able to type sl to sync live files to cvs, and sc to sync cvs files to live. I, personally, do all my changes in live, test them on the site, then "sc" to move them from live->cvs to generate the patch.

The parts of the instructions that assume you're generating patches as you go have to do with the fact that unless you want to muddle around with learning mercurial queues -- which I never have -- you will want to revert out your changes from the cvs directory after every patch you generate, or else each subsequent patch is going to contain all the other patches you did and the system will bitch at you when you try to upgrade to the newest code. (The command to revert changes is 'hg revert --all', and you need to be in ~/dw/cvs/dw-free or ~/dw/cvs/dw-nonfree when you run it.)

So, my own personal workflow goes:
1) make changes in live, test, get it working, yadda
2) "sl" to sync live files to cvs
3) "hg diff > ~/bugXXXX.patch" to generate a patchfile in my home directory
4) open the patch file, confirm it's got everything I need and doesn't have anything I don't need, upload to bugzilla, etc
5) "hg revert --all" in ~/dw/cvs/dw-free to revert the changes I just made in cvs
6) "sc" to sync cvs files (the unchanged versions) back to live and reset to the pre-patch state

This way doesn't take advantage of a lot of the things you can do with Mercurial, but given the complexity of the bugs I'm working on (ie, not much) it's just easier for me to do it that way, personally! If you want to learn Mercurial someone else would have to help out. *g*

[chagrined]

Well I had to tweak some things but I think I eventually sort of made this work. :) So thanks, that was helpful. One weird thing -- I noticed there's no ssldocs directory inside dw-free? At least I don't seem to have one. But there are two (identical) copies of the .css file I changed, one in htdocs/stc/widgets and the other in ssldocs/stc/widgets. But since my dw-free directory contains no ssldocs directory, the patchfile is only showing the differences in the htdocs version.

[ninetydegrees]

I have one! How do you get the new update page on one's hack?

[fu]

Edit $LJHOME/etc/config-local.pl, look for %LJ::BETA_FEATURES

Add this:

        "updatepage" => {
            start_time  => 0,
            end_time    => "Inf",
        },

(Inf == infinity ;-))

[ninetydegrees]

Oh I did that, I think and it doesn't seem to work. :/
This is what I have:

$ENABLE_BETA_TOOLS = 1;

and then:

%LJ::BETA_FEATURES = (
"journaljquery" => {
start_time => 0,
end_time => "Inf",
},
"updatepage" => {
start_time => 0,
end_time => "Inf",
},
);

What am I doing wrong?

[pauamma]

You first need to update the code on your dreamhack the usual way(*). Then, substitute your dreamhack's domain name (probably something like www.ninetydegrees.hack.dreamwidth.net) for www.dreamwidth.org in the "sign up for beta" URL. (See dw_beta or dw_news for the exact URL.) This should let you sign up for it on your dreamhack.

(*) If you did already, you may want to update it again anyway. There's been some substantial changes in the past week, IIRC.

[farasha]

Hi!

If one were interested in getting involved in bugfixes and development, but one's only code experience is outdated (HTML 3.0 and Java a few versions ago), which languages would you lovely people suggest one brushes up on?

[pauamma]

For front-end development:
- HTML4, CSS3, and the Template Toolkit templating language
- Javascript, the jQuery library, and the jQueryUI library

For backend development:
- SQL

For both:
- Perl

Also, feel free to pop up in #dreamwidth-dev if you need to discuss specifics, depending on which bug(s) you're looking at.

[denise]

HELLO AND WELCOME, we are kind of ridiculously full of the welcome wagon

In case you have not had ALL the links thrown at you yet:

* Dreamhacks, our hosted developer environments, because you don't want to try to install the code yourself
* The Wiki, especially Development and Getting Started
* The #dreamwidth and #dreamwidth-dev irc channels, on irc.freenode.net
* The Mercurial web frontend if you want to just read through the code instead of checking it out
* BUGZILLAAAAAAA, especially the unassigned effort-minor bugs