dw_dev_training | Question thread #3

Hi!

So I'm working on Bug 2886, and I've reached the point (with lots of help!) where I MIGHT have a working patch. And now I need to test it.

And I'm kind of stuck, to be honest.

How do I go about designing a testing plan for a bug like this? What kind of use cases do I need to take into account?

Thanks, guys!

Well, you've got 3 fields for each link, so it's a matter of testing how it works depending on what's filled:

- all 3 fields
- text + URL
- title + URL
- text + title
- text only
- title only
- URL only

Also, what happens if you put a URL in the title field? Text in the URL field? What happens if people use the URL to a picture? HTML in the title or text field?

Remember, when testing, you need to test both what you implemented, and what already existed, to make sure you didn't break it ;)

Good luck!

Still working on my script to post Pinboard bookmarks to Dreamwidth. (And much thanks to both you and

foxfirefey for your answers last time. I completely forgot to get back at you, my bad entirely. I'll try to be better this time :) )

I'm stuck on, I think, a fairly easy question, that I nevertheless can't find an answer to.

To use the Dreamwidth API (and the Pinboard one), I need the user's password. My script is PHP, and I tell people installing it to do so outside of the public folders, to reduce problems, but even though, I'm not particularly happy at having to store the password, not even encrypted, in the script. Everything I can find about storing passwords securely is about sites that ask you to login, but I do the opposite: I send the password outside. (Unless I'm a dunce and there is a totally different solution I didn't think of.)

I need to store the script on a web server, because the point is for it to run automatically without human input.

Anyone has the beginning of a solution for me?

A few thoughts off the top of my head, everyone feel free to expand, elaborate, add test cases...
- Links without a title (should still be possible to specify them, and they should still display correctly)
- Links with an empty title
- Links with a not-empty title that's shorter than the maximum allowed
- Links with a not-empty title that's exactly the maximum allowed
- Links with a not-empty title that's 1 byte longer than the maximum allowed
- Links with a not-empty title longer than that
- Links with a not-empty title with a Unicode character that straddles the maximum allowed (eg, if the maximum allowed is 40 bytes, what happens if a 2-byte Unicode character takes positions 40 and 41?)
- Links with HTML markup characters in the title (eg, innocuous-looking text"></a><script src="http://haxxor.example.net/pwned.js" /><a title=")
- Links with entities in the title, but no markup (like I &hearts; titles)
- Links with a single or double quote in the title

Finally, it shouldn't break any existing automated tests (which isn't to say that all tests in our test suite need to pass, but any test that passed should still pass, and any test that failed should fail in roughly the same way). Note that the test suite can be overwhelming, so you may want to leave that for a reviewer/committer to check.

Edited 2011-11-21 18:08 (UTC)

Ideally, the directory the password (and script) is in should be outside the Apache document tree (which may be what you mean by "outside the public folders). If that's not possible, you might be able to handle it without a gaping hole by having the password file in a separate directory that also contains a .htaccess that disables directory indexing and that denies HTTP access to all files by requiring authentication and giving an empty list of authentication methods, Thus, the password data will only be readable by locally-running scripts and won't leak out if someone tries to wget the password file by its URL. (This fails if an attacker manages to get their own code running on the server(*), but there's little you can do to protect from that no matter which approach you use.)

(*) If your script lives on a shared server, which is common for PHP, you should make sure that the server operator enforces separation between customers, eg by having a separate Unix account for each virtual domain's directory tree or something along those lines.

also, I should add, we have plans to implement OAuth, which will let you authorize an external service to have access to your account, etc. Someday!

Would it work to store the login cookies, instead of the passwords?

Would it work to store the login cookies, instead of the passwords?
I have no idea! XD I'm using the script as a learning process actually, and I'm making it up as I go along, probably reinventing the wheel a couple of times.

I'm going to keep your suggestion in mind for when I have time to tackle the documentation. Thanks!

outside the Apache document tree (which may be what you mean by "outside the public folders)
That's indeed what I meant.

you should make sure that the server operator enforces separation between customers
Mine does, and so does the host I recommend, but I'll have to add this to the requirements, then.

Thanks!

Reinventing the wheel occasionally leads to advances in wheel technology, so go for it. *g*

If you can bribe someone to help me with the user facing bits... >_> ( also uh, there should totally be a bug for OAuth stuffsies )

huh, I thought there was a bug for it. if not, open one!

And hey, if you open the bug, make it needs-design and lay out what the user-facing bits need to do/be/say/etc.

I could use a project to work on, and I know something about OAuth - I've interacted with Twitter's OAuth implementation as a dev, and implemented the client-side bits of it from scratch with no library (which I did so that I knew how it worked; for anything other than local tools, I'd obviously use a library), so... I'm willing to help!

[edit: Which is to say, I implemented it without an OAuth-specific library. I used libraries for URI escaping, URI parsing, HMAC-SHA1 encoding and base64 encoding.]

Edited 2011-11-21 20:24 (UTC)

I am only newly poking at the DW code and am new to most of this. I don't have enough understanding yet to ask interesting questions. But here is an uninteresting question I have about something!

In cgi-bin/bml/scheme/global.look, this is line 171:
GRIN=>{S}<grin>

Does that actually serve a purpose somewhere? It seems like all <?GRIN?> would do if used in a BML document is cause <grin> to print, which then wouldn't show up to the user, but wouldn't do anything else either. Is this just a silly programming easter egg of some kind, or does this actually do something I'm not seeing? (I also did a rgrep and couldn't find <?GRIN?> occurring anywhere.) eta: Oh, wait. I'm realizing now it'd make <grin> print, not <grin>. So the user would indeed see <grin>, I guess?

Edited 2011-11-22 14:21 (UTC)

Yes, they would indeed see <grin>. Also, the .look files for the legacy (pre-TT) skins have that unpleasant wrinkle that they define in all-uppercase stuff that's then referenced in all-lowercase elsewhere. (I got burned by that one when trying to clean them up early on in DW's life, IIRC.) In this case, though, it lr ooks like it's really not used, UPPERCASE, lowercase, or mixEd CAsE. (And it was already present in the initial version of that file Brad checked in on LJ, so it's really old.

Hum. Well if it's super old maybe long ago in LJ code it did serve a purpose, but now doesn't? Like idk maybe Brad had <grin> print out on some pages? That seems kind of silly. ;p

Also, it is very useful to know that about the case! I think I first searched for it in all-uppercase and then did a search with that "-i" non-case-sensitive option also just to be sure. :)

I would guess it predates LJ and is derived from something like IRC or Usenet slang. It would have been common at one time for people to type <grin> or <G> in a non-HTML chat environment, and they probably tried to do the same thing in early LJ comments. Unfortunately this theory is completely non-Googleable, so you'd probably have to find some ancient crusty netizens to ask about it.

Another question! I am poking at MoodThemeChooser.pm (and other things) in an attempt to change out places where tables were used for layout purposes, to use CSS positioning instead. In MoodThemeChooser.pm there is this variable $moodtheme_extra which takes its value somehow from the hook mood_theme_extra_content? And then another variable $show_special which is set based on that? (I am still just trying to figure out Perl, lol.) ANYWAY. I cannot figure out where mood_theme_extra_content comes from or what it is supposed to signify. I read the wiki page on hooks and the old LJ documentation on them but I'm still confused. And I did an rgrep search but couldn't find mood_theme_extra_content anywhere else. It's making me scratch my head, b/c I can't figure out how I should change the CSS version of the layout in relation to this element.

As far as I can tell, with the table, what it changes is that if $moodtheme_extra is false, then all the example "mood theme" images/descriptions (which are contained in <td> elements) are in ONE row. If $moodtheme_extra is true, then after every 2nd image it inserts a new <tr>. But not knowing what the heck mood_theme_extra_content is, I don't know why it's doing that or whether I should also do that. I wasn't sure also if maybe this is related to the width of the table, b/c I just told my CSS positioning to size the width of that element automatically, but ...if mood_theme_extra_content is related to mood themes that are extra super wide or something, maybe that is why it happened? (I'm positioning those elements with float:left so I could just add in a clear:left probably to the same effect.) (eta: or set a maximum width to the containing element to eliminate the need for this workaround?)

Anyway, confusion. I figure also for testing purposes it's important to know what that hook means so I can test the two different cases. On that end, also, I guess I have another question. How do I make a user account in my dreamhack a "paid account" so I can test paid features? Maybe there is an obvious answer to this somewhere but I couldn't find it on the wiki.

Edited 2011-12-03 20:16 (UTC)

That makes sense! Hmmm, although I wonder how one could use that from a user's end. Maybe earlier LJ comments didn't parse HTML in entries/comments like they do now? (Heh, I just tested to see if posting or commenting <?GRIN?> now would do anything, but ...nope, alas.) XD

For the paid account: go to /admin/priv while logged into the system account or someone with admin:*, give yourself the 'payments' priv, then go to /admin/pay and enter the username of the account you want to upgrade. the subsequent bits should be self-explanatory *G*

meanwhile, without looking into it too closely, i'm going to guess that mood_theme_extra_content is almost certainly some ljcom hook (the non-free lj code), and therefore we don't have it -- most (though not all) of the hooks defined in the code are. To look for where in the code it's referenced, you'd do:

rgrep -i mood_theme_extra_content $LJHOME/cvs/dw-free/* | more
rgrep -i mood_theme_extra_content $LJHOME/cvs/dw-nonfree/* | more

Since neither returns any results except the bit in the widget, it's pretty certain that it was a ljcom-specific hook (and now i'm curious; i looked it up (don't worry, you aren't supposed to know where to find it, I just happened to) and it looks like it has to do with sponsored mood themes, which we don't have. (There are a lot of hooks scattered around the code that LJ used to pull in ljcom-specific things like that -- some of them we've repurposed for other dw-nonfree things that are useful, some of them we just ignore.) So you can safely ignore it.

Yeah, I did
rgrep -i mood_theme_extra_content * > whatevertempfilename
while in the parent directory of my dreamhack, and those were the only places where it showed up.

What categorizes "ljcom" things? Are they all related to sponsored things, or are they code bits that are not open source or something? I wasn't really positive why the Mercurial repositories have both the "dw-free" and "dw-nonfree" branches (but the live directory doesn't?).

Since DW has no sponsored mood themes, in working on editing that module to use CSS positioning should I also edit it to not take that hook+corresponding variables into account? Rather than having lines of code that serve no purpose / do nothing? Or comment them out or something?

Also there were some hooks I was unfamiliar with also when I was poking at S2PropGroup.pm (which I've taken a break from to work on others instead). I don't remember them off the top of my head, but, it looks like that thing you linked to might be a good place to look up some of them. If there are some that don't appear to be registered anywhere in the DW build that I have, but I find them on that site, is it usually the case that they are ljcom things that are not in use? Should I double check any time I find one like that before assuming that it doesn't serve a purpose?

P.S. Thanks, it was causing me immense aggravation not being able to find that thing anywhere, ha.

Edited 2011-12-03 22:50 (UTC)

It's ... complicated! Basically, any time there's anything that makes assumptions about account structure, or references specific communities on LJ that might not exist on other sites, or assumes things that might not exist elsewhere, it's generally done with a hook. (For a good example of where hooks are used on DW for those reasons, check out htdocs/community/index.bml in dw-free and cgi-bin/DW/Hooks/Community.pm in dw-nonfree: "oh crap! I need to use DW-specific comms in the code!")

In general, if you see a hook being called in the code but you can't find any place where the hook is actually defined, it's almost certainly a ljcom hook. You can check for it in the ljcom code, but you can't copy or use that code.

(ljcom is the LJ equivalent of dw-nonfree: it is source-available (for comparison) but not Open Source and other sites can't use it. livejournal is the LJ equivalent of dw-free, which is freely usable anywhere. When we forked from LJ, we were only legally permitted to use things in the livejournal repository, not livejournal + ljcom the way livejournal.com uses, which is one reason why there are features on livejournal.com that we don't have.)

In this case, yeah, I'd just go ahead and remove that hook call in the mood theme widget if you're working there, since there's no reason to have it other than for something we don't use. If someone needs a hook there again in the future, they can make a new one.

Thanks for the explanation! It is especially useful b/c I had read the wiki page on hooks as well as the old LJ documentation on hooks, and I'd looked in places like cgi-bin/DW/Hooks to try to find where some of them were defined and was really frustrated when I couldn't find them anywhere. (lol, I did rgrep first and then when I couldn't find them was like, "BUT THE WIKI SAYS THAT HOOKS ARE DEFINED SOMEWHERE, THEY HAVE TO BE DEFINED SOMEWHERE!!!! AHHHHHHHHHHHHHH" *commence useless manual searching of file structure even tho I already confirmed it wasn't there with my rgrep search*)

I am curious: why is some of the DW code open source but the stuff in dw-nonfree isn't?

The stuff that's in dw-nonfree is basically the stuff that either a) we cannot legally redistribute (stuff where the licensing says that we can use it, we just can't redistribute it, and putting it in dw-free for other people to use is redistributing) or b) stuff that is part of our unique branding (the Tropospherical Red/Purple site schemes, our logos, etc). There's also some stuff in there that refers to DW-specific setup: the suggestions generator (posts to a specific DW comm that might not exist on other sites), some places on the site where we refer to DW specific comms (like the hook I mentioned earlier), some bits of our account structure (although most of the paid vs premium paid division is very "baked in" to the code, we do make at least a token effort to assume that other sites might have other account structures) -- stuff like that.

Thanks again for more explanation! :)

Anytime! No question too big, no question too small.

Question thread #3

Testing

Re: Testing

no subject

Re: Testing

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject