24 December 2015

Starting up a startup

I've been contracting for 3 years now, which fits nicely with my drive to have real customers and build business(es). But I still have an itch. I've now found something to scratch that itch (taking the metaphor too far, sorry about that).

I'm going to write about my latest adventure here, from the perspective of me as a coder who's aspiring to be an entrepreneur, so if you're interested in my personal journey on this then do add my blog to your rss reader etc. of choice. I'll try and tag them all with startup so you can filter if you aren't interested in my other ramblings :-)

This post tells the story how I got involved in this startup, and some of the technical choices made so far and what we have in place already.


So here's how it started for me:

I found out via twitter that the the startup britain bus was coming to Reading, so I went to say hi. They were a friendly crowd, and whilst there I found out that there's actually a new startup group in the area called ConnectTVT associated with the new co-working space called grow@greenpark over in green park just off M4 J11. That beats traipsing into the big smoke for me and is just the kind of melting pot I've been looking for. I know plenty of tech folks given that I'm a professional coder, but it doesn't seem sensible for me to start a business with people with the same skills and the same blindspots as me. None of my "normal" friends seem to have the same entrepreneurial itch that I do; I needed a broader pool. I went straight from the bus to grow to check it out and met some more lovely friendly people. A few weeks later (I think) I went back again for one of their startup grinds to see what was going on. There was a small group of interesting people with diverse interests and talents, from accountants to designers and a few tech folk. One of the people there, Richard, had an elevator pitch for a new take on the property market which made some sense to me and seemed well received by the group. All the people there offered their various advice and help. Richard has had experience in the property market that's led him to this insight and having mulled over the idea for quite some time was interested in how this concept might become something more real. Lacking the tech background to just go and build it himself he was interested in what kind of scale of task this might be and how to find people who could help. I offered to provide advice any time and passed Richard my contact details. I didn't at that point have any plans to get further involved.

I very much enjoyed meeting everyone, and went home with a head full of ideas and inspiration. I found myself still thinking it all through in the small hours that night, regularly grabbing my phone to make yet another note on the trello app detailing some idea about this property thing and how it could work, how you might get over the network-effect bump etc. Clearly this wasn't going away. So I messaged Louize who runs the meetup to get in contact with Richard and said it had been keeping me awake with ideas, and she got us in contact again.

After mulling the whole thing over for a bit longer, it occurred to me that there is no reason I couldn't do more than just provide advice, and maybe I should offer to provide a more concrete involvement. After all, I think the idea is great, and it's not like I'm busy making any of my other ideas I have into businesses yet. Richard would certainly be much better able to build at a much lower financial risk with a technical co-founder. So after a load more discussions around the idea, the odd trip to the pub and some discussions around practical arrangements that is now what we are doing.


How's it going?

The first thing has been to really understand Richard's vision, this has mostly involved talking lots. I set up a trello board for the startup, and that's been great for doing a brain dump of all the ideas and todos, and getting them into some sort of organisation, then moving them into "do later" or "needed for launch" kind of groupings. I've also pointed Richard at balsamiq mockups, which these days has a slick web based editor and presentation mode. I've used it in the past to great effect when helping stakeholders in other projects understand what they're really likely to get before all the expense of building the wrong thing. In this startup it's already been really useful for hashing out ideas of how the site might work and even for getting reactions and insight by way of putting it in front of people and seeing if they understand the concept and how you would use it if it were a real thing already. Inspiration for this process is taken from the excellent book Don't Make Me Think.

Hosting choices...

In terms of building something real, Richard bought a domain, so we have somewhere to put it. I spent forever over technical choices because basically you could use just about any technology stack, any programming language, cloud PaaS, IaaS or private servers etc. and in fact they would all work. There's actually nothing to completely rule out anything. Some great things have been built on top of apparently terrible technical choices (wordpress is lovely but php that it's built with is known for being buggy and hacky). There's much stock put in "cloud" these days, and it does have benefits such as infinite scalability (of your wallet), but it comes with an overhead of complexity and potentially vendor lockin (though that like everything can be mitigated). So call me a luddite but I've gone for an old fashioned linux VPS (virtual private server) where I can easily put a ceiling on costs till we have an income. Unlike so many modern startups that rely on farming people with free shiny stuff until they have enough they can magically make money off the back of them, burning VC cash all the while in the name of profitless growth, we're going to provide users with value, and charge them appropriately for that value, which naturally puts the brakes on the kind of explosive growth of free users that can overwhelm an ambitious startup's site. So I've made a decision at last, hunted around for a provider I could trust and bought some hosting. There's a lot to choose from so I needed a way to narrow down the field, especially given there's a lot of not very good and not very well supported VPS offerings out there. I thought there might be an advert in Linux Voice magazine, which is where I found my uber-geeky ISP A&A (check out the boss's blog if you want to know how far that rabbit hole goes), but there weren't any. So I hopped on #linuxvoice on freenode irc and asked in there, and sure enough the good folks at the mag said they'd been using bytemark and had been happy with their support. Sold.

So I bought bytemark's basic VPS offering, and now have a server on the internet that I can point everything at and run everything we need on no matter how quirky the needs. While we have a manageable number of users this should suffice, we're not planning any multi-million pound TV adverts any time soon. I expect if we're successful we'll have growing pains no matter what platform we're using, it's non-trivial to scale even if you're on a so-called scalable platform in the first place. There's lots of gotchas, unexpected interactions and bottlenecks in any system as it grows.

If it sounds like I'm trying to rationalise my decision that's because I am, I'm aware that whatever technical (and even non-technical) decisions are made at this earliest of early stages have rippling effects that are hard to reverse once you build more on top of them. However equally no matter which choices you make, none of them are truly irreversible, and there are myriad ways of tackling any of the problems that come your way in time. I've had the privilege of seeing teams that are running decent scale operations in both .net and ruby on rails, which are two very different platforms with different cultures and different sets of problems, but interestingly the difference in the overall scale of the challenge and how easy it is to get things done in the long haul was not noticeably different as far as I could tell.

Mailing lists and landing pages:

The first goal is to have a way to let interested people know when we've got things to show them, tell them the story, and when we need their help. That means a mailing list. Bulk email isn't for the feint hearted these days in the modern world of aggressive spam filters, SPF (no not sunblock), domain keys and increasingly rigid legal constraints as we try to fight outright spam on the wild west of the internet so I don't fancy doing it all myself. My go-to service for this is mailchimp, and I haven't been disappointed. But for this to work we need an email address, and that means MX records and pointing it at some mailserver somewhere. I did look into gmail as a possible shared mailbox but it doesn't seem to do what we're after, so now I have a server running anyway I can just use that. No artificial limits, no snooping, no unwanted advertising.

It turns out that bytemark provide a very nice customised build of debian called symbiosis, which has a bunch of useful services already installed, and a watchdog service that'll restart anything that's fallen over. Nice! It didn't take me much fiddling to get the basic web hosting to behave and get an email account on it set up. I then used the domain control panel to point the A & MX records at our shiny new server (by way of a bit of chaining of CNAMES which will be slightly slower for the first lookup but will be easier to maintain). I then had to redo the web and mail hosting because I hadn't realised that bytemark had made the server do multiple domain hosting using a neat folder based system, but that didn't take too long and it's all good learning time without customers breathing down one's neck!

I've now done a rudimentary holding page for the domain, with a sign-up form for the mailing list, which was goal number one. You can see it here: http://www.xchain.co.uk/. If you're interested in the concept or progress then please do sign up to the mailing list. It's easy to unsubscribe if you get bored of it ;-)

I've added google analytics to the site, which is very useful albeit slightly evil, so at least we'll have some idea if it's getting any traction as we start to spread the word (taking all stats on the internet as extremely approximate of course). I might switch to one of the more libre solutions at some point but that would require a bit more time and this is after-all a commercial venture and not an open source project. Much as I like working on open source for the greater good, I haven't yet managed to make it pay the bills. I think it's important to have metrics for a startup so you can tell if what you're doing to spread the word is actually having any effect, and what might be putting off potential customers.

We've sent out a mailing or two to the few people we've got signed up at this early stage (hello mum!) as much for practice as anything, and I'm impressed with mailchimp's collaborative capabilities and previewing system.

Tech stack

You may have noticed I still haven't entirely picked a stack, nothing I have so far has locked me in to a decision. I don't want to be paying for or locked in to Microsoft's world (and the associated treadmill of change), so although I do their stuff for a living I don't want to tie my own startup to them, so that's out. I'd prefer an open source stack as I think it matters whether you enjoy working on the platform you've chosen, and open source is just so much nicer to work with as a coder / dev-ops. Currently I'm thinking a golang based server, backed with postgresql (the industry's finest database) serving up a JSON/REST API to an AngularJS + Foundation frontend. This gives us scope to rework the different parts as needed, and to bolt on dedicated mobile apps later.

Web design

There's no design in the current holding page as I am no designer (frustratingly, but that's a whole other career path), and we're still working out how to pull design talent into the whole shebang in a sustainable way. Richard has put me onto webflow, which maybe will get us started.

My first learning of trying to find a freelance designer was that I need a "web designer" not just a "designer", being from the web world I hadn't appreciated the importance of the prefix and attracted all the wrong people.

Further reading...

Having this startup has encouraged me to broaden my reading and listening habits further. Here's some of things I've been catching up with:
 --

Please do drop me a line if you found this interesting in any way. tim@timwise.co.uk works best for me.


28 October 2015

Dealing with dodgy recruiter tactics

Updated version of this on my new blog: https://blog.timwise.co.uk/recruiters/

---

So I'm back on the contractor market again, and while many recruiters try and be reasonable, there are enough out there using underhand tactics to make life difficult and potentially jeopardise a good contract.

The one I plan to tackle this time round is "multiple submission", this is where you end up with your cv landing on a client's desk from two different recruiters. I gather this can be the nail in the coffin of a contract even if the client was keen - who wants to be stuck in the middle of two recruiters fighting over the commission?!

Principles


The first step is simple, which is don't agree verbally or otherwise to be put forward without first finding out who the client is, and make sure you keep track of who sent your CV where. I find Trello is a good tool for keeping on top of the fast moving and often fragmented information you get when hunting for contracts.

Recruiters are often hesitant to pass on the client's name, and this is understandable as if you were to go round them then they'd lose their commission which is how they earn a living. Worse, if you let the name slip to another less scrupulous recruiter then they may try and get your CV in first themselves without asking and in doing so sour the deal for everyone. (They often fish for names with phrases like "I need to know who else you've been submitted to in order to avoid duplication" - this is a lie, you can manage this fine yourself.)

To tackle this I recommend the following:
  • Never share their client's name with anyone else, especially other recruiters; make it clear this is a point of principle.
  • Prove your integrity by politely refusing to tell recruiters who ask, they might have liked the name, but they will be more likely to trust you in the future for respecting the wishes of other recruiters.
So that's the easy bit which I've been doing for some time...

Picture of an otter's head poking up in the sea.

Technical measures


There are unfortunately a bunch of particularly unscrupulous recruiters out there like sharks in the water who will without your permission, or possibly without ever contacting you, send your CV to their "client" (or just some poor manager they've found to spam with CVs). You can't tell them not to if they've never asked you, and it can still ruin a deal. So what to do about that?

If you're looking for contract work, you really need good exposure, so having your CV all over the place like leaves in the wind is not a bad way of getting the word out there (bear in mind I'm looking for work for my contracting company, which is a different ball-game to finding that perfect permanent role). It's actually pretty hard to control at all if you're dealing with recruiters because as soon as you include your CV in a response to posting on JobServe it'll be dropped straight into that recruiter's pool of CVs, and some recruiters even pool CVs between them using services like iProfileUK.

My plan of action is this: having just brushed up my CV, it will contain the following text (as does my LinkedIn profile):
Recruiters: this CV is not authorised for distribution to your clients. Please contact me for permission to represent me and for a separate copy containing an authorisation code & gpg signature. Thanks. To prospective clients, if you receive my CV without these then I haven't given permission to be represented.
Okay, so far so good, but any unscrupulous recruiter could just strip that out and send it anyway, and how would I be able to make a client comfortable that they can tell this scumbag to jog on?

So before I agree to be represented by a particular recruiter to a particular client (which I have no particular objection to if they've found work I otherwise couldn't have found), I will be needing the client's name. I will add this to a list of who has authorization to send my CV where; I will then generate a customised CV (via mail merge) with explicit permission to represent me to this specific client included in it, and a note that any CVs received without this are unauthorized by me.

To prevent a recruiter cottoning on and just generating this themselves I will then be gpg signing the result, which can then be checked against my public key, proving that it did indeed come from me (assuming my pc hasn't been hacked of course but I haven't heard of any recruiters going that far, if they could they'd probably be security consultants instead of recruiters!). 

My public key


My current public key for tim@timwise.co.uk is available on the public keyservers, or you can download it: my public key on dropbox. Primary key fingerprint: 74D4 2A4C 9055 07C5 4A7E  3C9C 26C6 E087 28CD F8EA.

Technical details

Using libreoffice calc, codes are generated with the following:

A cell containing allowable characters in the codes as text: 0123ABCDEF etc for this example this is in cell "F1".

A row for each authorized representation containing this formula to generate a unique authorisation code: =CONCATENATE(MID($F$1,RANDBETWEEN(1,LEN($F$1)),1), [repeat the "mid" clause once for each digit of the code to be generated] )

There's then another concatenation cell to generate the message to add to the CV. This is then copy-pasted into a file, and signed with gnupg on the command-line of my linux box. Here you can see the signing happening, followed by verification that the file is signed properly (as the client might do if they suspect a recruiter is reusing my CV for without my permission).

$ ./sign.sh
#!/bin/sh -v
gpg --clearsign authorization.txt

You need a passphrase to unlock the secret key for
user: "Tim Abell <tim@timwise.co.uk>"
4096-bit RSA key, ID 28CDF8EA, created 2015-01-20
$ ./verify.sh
#!/bin/sh -v
gpg --verify authorization.txt.asc
gpg: Signature made Thu 29 Oct 2015 01:02:52 GMT using RSA key ID 28CDF8EA
gpg: Good signature from "Tim Abell <tim@timwise.co.uk>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 74D4 2A4C 9055 07C5 4A7E  3C9C 26C6 E087 28CD F8EA
the contents of the signed file is then copy-pasted onto the end of a CV and sent off to the recruiter to relay to their client.

Examples

I've uploaded a couple of files for you to try out verifying, one with the original message, and one with a forged message where the client's name has been changed. See if you can figure out which is which:
Here's an example signed authorization with a valid signature for the end of my CV (mail merge fields in bold):
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tim Abell has given permission for Test Recruiter Ltd to pass on this CV to Some Client on 28 Oct 2015; auth code 662QP93XP4. Any copies of my CV received without a valid signature have not been authorized for distribution.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJWMXA8AAoJECbG4IcozfjqtKYP/0DGQBU64kRsAqmnkpRkJEIo
5gu1RVvozLE1AjE+/l7cq3LHo9jr3FUeigFD+DRcxVnfUT4pmyWHJYaqiYkm6MFf
dx/3DAwj9g1sUOr4Ef0CN3MuSiP0HqrNanRCao0qxvCw9GJlKO+vTJlo8ZZMt+V0
ImNxAIwjwKKLTvaLc7nxAlyjS8ZVuykBO5BTu5NnP4oDD359g1MEa6wI61et6bmj
jvQiSQpqJnHxs+EN1QvrkVuDLZUT2OOQxwnKJV3WfI87YBuaMH/Eguor5NIQvnXT
Z0vm5WbVgbadvpL5MP722EiNx+Ab4VCPJmEkrarH24em8f+JbZsvU753rZ4zgbOO
TtO71Bn2DtjvgT6cVEytDbpJcl0pAhCjbSYkFCXOQYeP9lcBopbdTJ6nRcFcFEFK
wlMcf+Cw/9WnYwyPMaWJi2zggJjLoWX2vRV5yUVa7y92pi9Fi2ow6Cru8WgXz/WW
FMqOTDdRcYbDWCcDT7vc3oW0yfIkaCxnSm1jcg+/VUr1X/9zuNxcV+qRPUJ15fYR
ii+cw/hIEqxn0tr+6eUsDmKdmS87PGGpo0wlGf3z7peytITIPf9y8bs2+N8ok6kA
8IXtZXqzYO3qnaIw2JUQidm/0zQ8xz4UNsSrqN2ZsxDtqGeDgHAn7+yFH7dvfTTV
ndYHkrMcYt1Yn5AmIsgZ
=tFS7
-----END PGP SIGNATURE-----

27 October 2015

Sql Data Viewer - preview release

Hello dear readers,

I've started working on a data brower for relational databases that allows you to click through ids with foreign-keys to get around.

Take a look at http://www.timwise.co.uk/sdv/ and if you think you might be interested then please do sign up to my mailing list, and let me know if you have any feedback.

This started as a holiday exercise in learning google's go programming language, but is intended to become a fully-fledged software product available to buy.

Tim

12 August 2015

100 reasons I hate ssrs

A slightly tongue-in cheek hit-list of nasty things and vague hand-waving opinions on what makes microsoft's sql server reporting services (ssrs) such a pig to work with.

I don't really know of anything better so this is mostly just pointless ranting; but I'll justify it to myself by saying at least you'll know what you're getting into if you've read this before you start. SSRS seems to be more "death by one thousand paper cuts" than completely broken, so it's not so easy to say "it's shit, shalln't use it" like any good prima-donna developer would. Sorry I mean rock-star (recruiter speak). It also offers a few features that would be pretty hard to code from hand cost-effectively in something like asp.net mvc, such as user editing, multiple export formats, scheduled emails, and some of the ways you can cut-and-shut the data in the reports.

I make no apologies for the colourful language, it's a representation of the emotional side of having to use this heap of crap.


  1. http://www.allenkinsel.com/archive/2013/01/adventures-in-ssrs
  2. doesn't bind to a port like a normal fukcing service
    1. http://blogs.devhorizon.com/reza/2008/10/20/say-goodbye-to-iis-say-hello-to-httpsys/
    2. http://www.iis.net/learn/get-started/introduction-to-iis/introduction-to-iis-architecture#Hypertext
    3. https://social.technet.microsoft.com/Forums/sqlserver/en-US/f2586aca-78fe-40d6-9bcd-5151bac7136f/role-of-httpsys-in-ssrs-2008-?forum=sqlreportingservices
    4. http://blogs.technet.com/b/andrew/archive/2007/12/04/sql-server-2008-reporting-services-no-longer-depends-on-iis.aspx
    5. http server api (aka http.sys)
      1. https://msdn.microsoft.com/en-us/library/aa364510%28VS.85%29.aspx?f=255&MSPPError=-2147217396
    1. list reservations:
      1. `netsh http show urlacl`
  3. auth in reporting https://msdn.microsoft.com/en-us/library/ms152899.aspx
  4. reports in VS
    1. http://curah.microsoft.com/22200/create-ssrs-reports-using-visual-studio
    2. https://msdn.microsoft.com/en-us/library/ms173745.aspx
  5. "Explicity add new role assingment for the account you are using and check every box in sight" ~ a.n. colleague.  lol
  6. ignore the .rdl.data files with git.
    1. http://stackoverflow.com/questions/3424928/in-ssrs-is-there-a-way-to-disable-the-rdl-data-file-creation#3425429
  7. no folders.
    1. https://connect.microsoft.com/SQLServer/feedback/details/487106/allow-sub-folders-in-ssrs-projects
    2. http://stackoverflow.com/questions/3309002/visual-studio-for-ssrs-2008-how-to-organize-reports-into-subfolders-in-solutio
  8. renamed a dataset, nothing fucking broke!!!!!!!!!!!!!!!!!!!!!!!!!!! even though there are reports that depend on it. On editing the report's dataset list you can see clearly "not found", but yet it still runs. what in the blazes is that all about?
    1. caching in the report editor http://stackoverflow.com/q/3424928/10245
    2. kill the .data cache files http://stackoverflow.com/questions/3424928/in-ssrs-is-there-a-way-to-disable-the-rdl-data-file-creation
    3. git clean -xfd
    4. fuck
    5. https://social.msdn.microsoft.com/Forums/sqlserver/en-US/0aa81692-352f-4c1f-a0e3-95fe6c0797ca/cachedataforpreview-in-rsreportdesignerconfig-not-honored
    6. https://connect.microsoft.com/SQLServer/feedback/details/468482
    7. it's the `bin\` folder, not the .data files. Still, fuckkkk.
  9. to get from a report to a db you go, report > report dataset > shared dataset > db, but db is defined in the shared dataset with another name, which can be pointed to a shared data source, which is also named. and *that* data source actually has a connection string
  10. committing to tfs failed half way through because vs had locked a bunch of files I didn't even have open
  11. found a param with <Value>=Microsoft.VisualBasic.Strings.Join(Parameters!Stages.Label, ", ")</Value> - wtf.
  12. function overload matching warning wouldn't go away till I closed the sln
  13. localisation is a bitch
    1. http://stackoverflow.com/questions/10953629/how-to-change-ssrs-2008-locale etc
    2. http://blog.ponytailbob.com/2007/10/multi-language-tips-in-ssrs.html
    3. http://blogs.msdn.com/b/sriram_reddy1/archive/2012/01/09/localization-in-ssrs-reports.aspx
    4. https://support.microsoft.com/en-gb/kb/919153
    5. http://www.keepitsimpleandfast.com/2011/09/localization-of-your-ssrs-reports.html
    6. why you no use User!Language??
  14. Visual Studio 2013 crashed. hard. while cancelling new report param
  15. adds 00:00:00 to date fields from sql server. duuuuuuuuuuuuh (goes via .net datetime internally, but even so, not friendly)
  16. changed date format, looks fine in VS, but no change in report server. wuh? deploy all
  17. no auto-sizing of cols http://stackoverflow.com/questions/7851045/ssrs-tablix-column-cangrow-property-for-width
  18. no nulls in multi-value http://www.keepitsimpleandfast.com/2012/03/how-to-pass-null-value-to-multi-value.html
  19. need dirty hack to show "all" rather than full list
    1. http://www.mssqltips.com/sqlservertip/2844/working-with-multiselect-parameters-for-ssrs-reports/
    2. http://stackoverflow.com/questions/12917261/optional-multi-valued-parameters-in-ssrs
    3. http://www.bi-rootdata.com/2012/09/efficient-way-of-using-all-as-parameter.html
    4. http://blog.ponytailbob.com/2007/10/2-shortcomings-of-multi-valued.html
  20. some fucking horror I've yet to encounter (querystrings) http://stackoverflow.com/questions/512105/passing-multiple-values-for-a-single-parameter-in-reporting-services
  21. it has a fucking canvas size that will push over to 2 pages
  22. the font kerning on a print is massively different to on web / design view
  23. sorting
    1. http://stackoverflow.com/questions/9254604/why-does-my-sql-server-reporting-service-ssrs-report-appear-to-re-sort-the-d
  24. "Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index " from editing xml. thanks for the error info. fuckkkkkers
  25. the ssrs gui editor is a flaky piece of shit
    1. doesn't select the right fucking textbox in the props window
    2. had to restart visual fuckigjn studio
    3. grrr
  26. the underlying xml is fucking horrific
  27. <rd:Selected>true</rd:Selected> ----- what in the fucking blazes is that doing in there?
  28. how do you deploy without connecting visual studio to production server? you fucking don't hahahahahaa
  29. powerhell https://gist.github.com/timabell/7e3019bd2de802f0b259
  30. ssbi install croaked - https://support.microsoft.com/en-us/kb/2800050?wa=wsignin1.0
  31. [09:44:53] john doe: Tim can I have a .bak file of ReportServer$MSSQL2012TempDB which the stupid software seems to be unable to operate without even though it has Temp in the database name implying it will rebuild itself (at least that's what it implies to me)
    1. [09:47:48] Tim Abell: (facepalm)
    2. [09:47:49] Tim Abell: sure
    3. [09:48:08] Tim Abell: I did wonder, and then I thought, no they couldn't possibly need that
  32. dropdown doesn't work in firefox
  33. no debugging http://stackoverflow.com/a/14068447/10245
  34. Warning : The text box ‘appliedFilters’ and the image ‘urLogo’ overlap. Overlapping report items are not supported in all renderers.
  35. the only options for DRY in reports suck balls http://www.3pillarglobal.com/insights/tips-tricks-ensure-consistency-sql-server-reporting-services-reports
  36. http://harmful.cat-v.org/software/xml/ xml is a terrible format anyway
  37. layout is in inches
  38. you can change the layout to cm
  39. it stores different metrics (cm/in) for each element, wtf, pick a unit
  40. reflowing nicely is impossible
  41. layout engine is as intelligent as a piece of paper and a pen. x,y is all you get.
  42. http://www.webapplicationsuk.com/2010/07/word-html-renderer-ndash-the-road-to-hellhellip/
  43. https://connect.microsoft.com/SQLServer/feedback/details/540183/supported-rdl-object-model-rdlom
  44. this is the kind of bullshit that counts for helpful content on the net https://social.msdn.microsoft.com/Forums/en-US/86205ca4-13d0-4ca6-84f1-79797616f0f4/exclude-null-values-from-sum-and-avg-calculation?forum=sqlreportingservices - =sum(forum_format * quality_of_community) = errorrrrrrrrr
  45. multiple rdl xml schema in the same fucking project, completely different xml structure
    1. 2005 generated with "new report wizard" in VS 20-fucking-13: <Report xmlns="http://schemas.microsoft.com/sqlserver/reporting/2005/01/reportdefinition" xmlns:rd="http://schemas.microsoft.com/SQLServer/reporting/reportdesigner">
    2. 2008 <Report xmlns="http://schemas.microsoft.com/sqlserver/reporting/2008/01/reportdefinition" xmlns:rd="http://schemas.microsoft.com/SQLServer/reporting/reportdesigner">
    3. 2009 from https://technet.microsoft.com/en-us/library/cc627465%28v=sql.105%29.aspx - <Report xmlns:rd=http://schemas.microsoft.com/SQLServer/reporting/reportdesigner xmlns="http://schemas.microsoft.com/sqlserver/reporting/2009/01/reportdefinition">
    4. 2010 <Report xmlns:rd="http://schemas.microsoft.com/SQLServer/reporting/reportdesigner" xmlns:cl="http://schemas.microsoft.com/sqlserver/reporting/2010/01/componentdefinition" xmlns="http://schemas.microsoft.com/sqlserver/reporting/2010/01/reportdefinition">
    5. how many fucking versions??!
    6. http://stackoverflow.com/questions/15539859/what-is-the-difference-between-rdl-2008-schema-and-rdl-2010-schema-feature-wise
    7. http://stackoverflow.com/questions/9974179/is-there-a-new-version-of-rdl-schema-for-sql-server-2012-denali
    8. http://stackoverflow.com/questions/29951653/ssrs-2008r2-visual-studio-2008-and-2008-and-2010-schemas - how to not end up with old schema?!
    9. https://social.msdn.microsoft.com/Forums/sqlserver/en-US/f4d14548-c592-4d8d-8185-ca683c421649/2010-schema-with-visual-studio-2010?forum=sqlreportingservices
  46. how do you upgrade a report schema? install a massive chunk of fucking sql server http://stackoverflow.com/questions/13170608/upgrade-my-rdlc-schema-from-2008-01-to-2010-01
  47. [17:54:48] john doe: Btw have you noticed that in Print Layout view the header doesn't expand if any of the textboxes have auto-grown?
[17:55:14] Tim Abell: that's because ssrs is a piece of shit from 1990
[17:55:28] Tim Abell: and it thinks A4 is the ultimate display format
[17:55:47] Tim Abell: you just have to guess how much space you'll need
  48. the ordering of the xml in the proj file is unstable causing diff noise
  49. subreports, icky http://bhushan.extreme-advice.com/subreport-in-ssrs/
    1. or nested tables http://stackoverflow.com/questions/11335655/filtering-nested-data-regions-in-ssrs
  50. some things can only be achieved with subreports, and they have to be deployed separately from the main report, meaning they can get out of sync. enjoy the fear of not knowing if you'll break something else when you upload your new version of the subreport you depend on
  51. no support for "time" data type http://stackoverflow.com/questions/3846378/displaying-time-in-reporting-services-2008
  52. The "View Report" button next to the parameters when running a report in VS is *always* greyed-out, even though it actually works.
  53. wow that's mental, hidden reports show in details view and not in tile view in the ssrs web ui
  54. the report editor has a copy option for report items, but no paste, so you can't duplicate reports
    1. actually you can, but only if you know the keyboard shortcuts. 0_o - ctrl-c ctrl-v
  55. duplicating a report on the filesystem, and then using "add existing item" to include it puts it at the end of the list... until you rename it and then it's moved into alphabetical order causing a spurious diff. should have put in the right place in the first place. grr.
  56. using the cursor keys to move textboxes around is so laggy that I overshoot every single time
  57. the editor popups in visual studio are modal, so you can't refer to anything else
  58. and there's no maximise button so you have to drag the fiddly border to make it bigger
  59. the report editor hasn't heard of ctrl-c or ctrl-v, have to use ctrl-Ins / shift-Ins instead
  60. border rendering / precedence is a fucking mess. set some borders, your report will look like a two-year-old coloured it in, and how it looks changes depending on the zoom level.
  61. you have to use VB to do alternate row colours - http://stackoverflow.com/questions/44376/add-alternating-row-color-to-sql-server-reporting-services-report
  62. if anything goes wrong with an expression all you get is "#Error". Helpful. E.g. http://stackoverflow.com/q/9144312/10245
  63. this one time, at band camp, I edited a report definition in VS and it refused to save the new definition to disk (ctrl-shift-s, ctrl-shift-s!!). wtf. Restarted VS and all the changes were gone.
  64. report editor silently adds new parameters to the report when you add new parameters to the sql. seriously. fuck off.
  65. RSI-inducing UI for editing the reports. click click click clickity click
  66. the sql editor has only a single undo. like ye olde notepad.
  67. in the editor, you can right-click copy, you can't right-click paste. wtf. ctrl-v does paste though. wtf again. I know, I already said it, but it's reaaaaaly shit
  68. the field list on a dataset is ordinal, allowing you to mismatch the select in the sql from the list of fields in the dataset and not notice
  69. how do you align a textbox on the page?
                  <Top>0.82546cm</Top>
                  <Left>0.07309cm</Left>
                  <Height>0.88964cm</Height>
                  <Width>2.3025cm</Width>
    fuck you!!!!
  70. one goddam cell in the underlying format:
                    <TablixCell>
                      <CellContents>
                        <Textbox Name="qty">
                          <CanGrow>true</CanGrow>
                          <KeepTogether>true</KeepTogether>
                          <Paragraphs>
                            <Paragraph>
                              <TextRuns>
                                <TextRun>
                                  <Value>=Sum(Fields!qty.Value)</Value>
                                  <Style />
                                </TextRun>
                              </TextRuns>
                              <Style />
                            </Paragraph>
                          </Paragraphs>
                          <rd:DefaultName>qty</rd:DefaultName>
                          <Style>
                            <Border>
                              <Color>LightGrey</Color>
                              <Style>None</Style>
                            </Border>
                            <PaddingLeft>2pt</PaddingLeft>
                            <PaddingRight>2pt</PaddingRight>
                            <PaddingTop>2pt</PaddingTop>
                            <PaddingBottom>2pt</PaddingBottom>
                          </Style>
                        </Textbox>
                      </CellContents>
                    </TablixCell>

Contributed (thanks, a nice surprise!):

71. Report width limited to 455in (even if I'm hiding columns using parameters against the Visibility column filter).
72. NO DYNAMIC COLUMN CREATION (ridiculous!)
73. Selection of multiple columns and setting attributes is ridiculously flaky. This is because I wanted to reduce the column width and font to comply with Point 71 !!
74. We have to restart Reporting Server services frequently or our charts won't show up. Eh?
75. Cut and paste columns? Nope!

More recent things:

76. Disappearing "Report Data" menu: http://stackoverflow.com/a/28883272/1024
77. SSDT for VS2015 upgrades reports to 2016 schema as soon as they're opened (WAT?!)  and ignores the TargetServerVersion being set to <=2014 http://stackoverflow.com/questions/37816216/deploy-of-a-report-with-ssdt-2016-generates-error what if we haven't upgraded our production server, hmmm?
78. Okay so when you run a build, VS2015 then downgrades the rdl to the right schema to match the project's target server version before putting it in the build folder. What could possssssibly go wrong. 0_o   Complexity++
79. Install SSRS in only 41 easy steps https://thecodeattic.wordpress.com/category/ssrs/
80. Build > "Skipping 'SomeReport.rdl'. Item is up to date." - No it isn't, I've deleted it from the friggin bin folder. So clearly it has some stupid cache of what's it thinks is on disk rather than, oh I don't know fucking checking the disk. Sheesh. If it's that optimized why is everything still so damn slow?!
81. Building a project with lots of reports is slow. Even if nothing changed.
82. Intermittently get "[rsInvalidReportDefinition] The definition of this report is not valid or supported by this version of Reporting Services. The report definition may have been created with a later version of Reporting Services, or contain content that is not well-formed or not valid based on Reporting Services schemas. Details: Data at the root level is invalid." - I am not alone. https://connect.microsoft.com/SQLServer/feedback/details/2988044/randomly-get-rsinvalidreportdefinition-when-previewing-report - A rebuild fixes it for me, for a while at least.
83. Sometimes when you edit an embedded dataset it completely fails to persist any of your changes to disk. And when you close the report they are lost. Handy. Thanks fuck for git.
83. Assumes you've never heard of source control and creates numbered backups of report files (notably on auto-upgrade). Also has messages like "delete will permanently delete this thing" - no it won't I have source control; I wonder if the SSRS know what that is.
84. I've never managed to crash VS2015 so many times in one day. Omg you didn't click there did you? I wasn't ready! *crash* ... *again*
85. Change the files on disk and the preview window often doesn't notice
86. Preview window silently fires a 'build' of the reports. Sometimes.
87. Generates broken Shared-datasets http://stackoverflow.com/a/38753141/10245
88. If you break the xml of a shared-dataset the entire project will fail to load. w-t-f.
89. Renaming datasets etc just breaks everything rather than updating references.
90. The .rptproj file has a <state> tag at the top which is base64 encode xml (WAT?! xml in xml. eerrrrr), which is information about source control (arse-about-face or what! source code that controls the *source-control* [that controls the source, that controls the source-control that controls the source...]; even the words are circular!!). In the base64 you'll find a <SourceControlInfo> tag.

<Project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" ToolsVersion="2.0">
<State>$base64$PFNvdXJjZUNvbnRyb2xJbmZvIHhtbG5zOnhzZD0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEiIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIHhtbG5zOmRkbDI9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDAzL2VuZ2luZS8yIiB4bWxuczpkZGwyXzI9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDAzL2VuZ2luZS8yLzIiIHhtbG5zOmRkbDEwMF8xMDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDA4L2VuZ2luZS8xMDAvMTAwIiB4bWxuczpkZGwyMDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDEwL2VuZ2luZS8yMDAiIHhtbG5zOmRkbDIwMF8yMDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDEwL2VuZ2luZS8yMDAvMjAwIiB4bWxuczpkZGwzMDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDExL2VuZ2luZS8zMDAiIHhtbG5zOmRkbDMwMF8zMDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDExL2VuZ2luZS8zMDAvMzAwIiB4bWxuczpkZGw0MDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDEyL2VuZ2luZS80MDAiIHhtbG5zOmRkbDQwMF80MDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDEyL2VuZ2luZS80MDAvNDAwIiB4bWxuczpkZGw1MDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDEzL2VuZ2luZS81MDAiIHhtbG5zOmRkbDUwMF81MDA9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vYW5hbHlzaXNzZXJ2aWNlcy8yMDEzL2VuZ2luZS81MDAvNTAwIiB4bWxuczpkd2Q9Imh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vRGF0YVdhcmVob3VzZS9EZXNpZ25lci8xLjAiPg0KICA8RW5hYmxlZD50cnVlPC9FbmFibGVkPg0KICA8UHJvamVjdE5hbWU+U0FLPC9Qcm9qZWN0TmFtZT4NCiAgPEF1eFBhdGg+U0FLPC9BdXhQYXRoPg0KICA8TG9jYWxQYXRoPlNBSzwvTG9jYWxQYXRoPg0KICA8UHJvdmlkZXI+U0FLPC9Qcm92aWRlcj4NCjwvU291cmNlQ29udHJvbEluZm8+</State>
...

decoded "state" contents:

  <SourceControlInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http://schemas.microsoft.com/analysisservices/2010/engine/200" xmlns:ddl200_200="http://schemas.microsoft.com/analysisservices/2010/engine/200/200" xmlns:ddl300="http://schemas.microsoft.com/analysisservices/2011/engine/300" xmlns:ddl300_300="http://schemas.microsoft.com/analysisservices/2011/engine/300/300" xmlns:ddl400="http://schemas.microsoft.com/analysisservices/2012/engine/400" xmlns:ddl400_400="http://schemas.microsoft.com/analysisservices/2012/engine/400/400" xmlns:ddl500="http://schemas.microsoft.com/analysisservices/2013/engine/500" xmlns:ddl500_500="http://schemas.microsoft.com/analysisservices/2013/engine/500/500" xmlns:dwd="http://schemas.microsoft.com/DataWarehouse/Designer/1.0">
  <Enabled>true</Enabled>
  <ProjectName>SAK</ProjectName>
  <AuxPath>SAK</AuxPath>
  <LocalPath>SAK</LocalPath>
  <Provider>SAK</Provider>
</SourceControlInfo>

Enough xml namespaces for you?

91. The state base64 changes all the time causing diff noise.


I know I can't count, but if ssrs is going to make so little effort I don't see why I should. And to be honest "100" seemed a lot more like comic exaggeration when I titled my tomboy note which only had 5 or 6 grumblings in it, I wasn't actually expecting to get within spitting distance of the original number!


Doesn't mean I won't use it again mind, just don't promise to like it.

If you liked this, you might also like the ssrs deployment tool I sometimes look after: https://github.com/timabell/ssrs-powershell-deploy (mostly not my work, just pulled together a bunch of contributions).


12 January 2015

LibreOffice mail merge - "data source 'SOURCE' was not found"

So another year on, LibreOffice 4.2.7.2 (via Linux Mint 17.1) still has a dog's breakfast of a mail merge feature, hey ho, hopefully it might actually get fixed following the fork from OpenOffice and the change in contribution methods.

Anyways

So I've moved machines, copied my files across and for some reason my mail merge has soiled itself and now bleats "data source 'SOURCE' was not found" which is as unhelpful as it is infuriating, especially given that the "check connections" button is exactly the wrong place to look for an answer.

Turns out you actually get this if even just a single field in your document is 'broken'. How do you tell which ones are broken? Well you have to change them all to just be sure. Sigh.

The fix for me today was as follows (though with such a messy feature there's unlimited ways it can break):

  1. Hit F4 and check that your connection to the spreadsheet actually exists and works, and unbreak anything you find therein. While you're in there you can marvel at how it requires a whole other file (.odb) just to remember how to get to a spreadsheet. (See below for fixing this)
  2. Turn on the field names so you can see what the f*** is actually going on with "View > Field Names (ctrl+f9)" which will show you the fully qualified field name, which might even be completely wrong. You can now see that for whatever reason (insanity?) it embeds more than just the field name at the field place-holder.
  3. And finally the way you actually fix the broken fields it's failing to tell you about actually lies under the menu item "Edit > Fields", where you can change all the broken references one at a time to the correct place.
  4. For bonus points, if it the field looks right but is silently broken somehow then you have to change the field to something else, hit okay, and then change it back again for anything to actually change, which is annoying if you have a lot of fields.
Fragile much?

Another fix I've just discovered is you can rename your data source to match the name defined in the fields (assuming they're all the same) and it'll start working again.

Fixing the .odb file

If you're stuck on point 1, here's how you fix it, also completely non-obvious and full of apparent dead-ends and dubious information.

  1. Give up on trying to do this in writer, it doesn't seem possible, in spite of false hope from the data sources tool, it only allows you to select .odb (database) files, not spreadsheets.
  2. Open up "libreoffice base", which pops open the database wizard
  3. Choose "connect to an existing database"
  4. In the dropdown choose "Spreadsheet"
  5. Next
  6. Browse for your spreadsheet
  7. Next
  8. Leave "register database for me" selected
  9. Leave "open the database for editing" checked
  10. Finish
  11. It prompts to save the new database (.odb), I suggest saving it in the same folder as the spreadsheet to save future confusion.
  12. You now have the database open in "base", you should see your spreadsheet sheets listed as tables
  13. Open a table (i.e. a sheet) and check you can see the spreadsheet contents
  14. Close "base", saving changes
  15. Return to your writer document
  16. Open the data sources again (F4), you should now be able to browse your spreadsheet via your newly created database.

Simpler than getting planning permission out of a vogon. :-/


Hope that helps some other poor open source die-hard who has work to do.


Useful refs: