Case notes investigated by the Parliamentary Ombudsman
The complaint
An applicant for a training subsidy, felt aggrieved that the Training Subsidy Scheme ETC (TSSA) Appeals Board did not uphold his appeal against the decision of the Selection Board in respect of his application requesting a training subsidy for a programme of study in Digital Media on the grounds that he did not have ‘A’ Levels. He argued that his IT relevant qualifications, work experience and motivation were more relevant than ‘A’ Levels. He also stated that the TSSA Appeals Board was giving more points for ‘A’ Levels when the course was in Digital Media, an area in which he had more qualifications.
Findings
There were 262 applicants for the Scheme of which 257 applicants were interviewed. The criteria that were adopted were essentially in line with those announced in the Scheme itself, namely:- academic merit; suitability (including performance during the interview; relevant experience and motivation); relevance of applicant’s choice of study/impact in the development of Malta; and the availability of much expertise in the economy.
Complainant appealed his interview result, and specifically as to how the Interviewing Board had evaluated his skills, education and work experience. In respect of his academic merit he referred to his nine (9) ‘O’ Level certificates besides a number of IT qualifications.
The terms of reference of the Appeals Board are listed in the European Social Fund document entitled:
“Project ESF 1.33 & ESF 2.4
Training Subsidy Scheme MCAST & Training Subsidy Scheme (Academic)
TSS Selection Board – Digital Media (TSS – SB –DM)
Malta 2007-2013”
Article 3 of this document dealt with Appeals, and stated as follows :
“Under no circumstances shall the Appeals Board change the score of applicants deriving from interviews by the TSS–SB-DM, nor change the ranking of an applicant as a result of such score”.
It resulted that in assessing candidates, the interviewing Board “…gave an individual and not an automatic assessment to each certification (at all levels) based upon the response during the interview as well as the capacity of the applicant to prove his or her achieved competence in relation to the aspired Higher Qualification. Similar qualifications were therefore given different marking based on the individual’s response to his or her aspired higher qualification and in relation to the acquired academic experience for that qualification. The Board made full use of the interview to assess not just the paper qualification (at face value) but also the academic merit of individuals who sought to link their previous academic experience with their prospective qualification. The Board gave more weight to the individual’s grasp of the academic area in relation to the prospective qualification than to a simple evaluation of a qualification at the level of the Malta Qualifications Framework. Although this was felt to be time consuming, the Board wanted to ensure that academic merit is evidenced by an individual’s performance during interview (vis-à-vis his or her qualification) and not just paper qualifications.
Therefore, besides the scores on the qualifications already obtained by the applicant, the Interviewing Board gave weight to other factors including:
- the grade obtained in the qualifications possessed (example: persons with higher grades were given higher scoring…);
- the aptitude of applicants during the interview (the Interviewing Board asked a number of questions on which the applicants were assessed and those with the most innovative and creative ideas were given higher scorings even though they may have possessed similar qualifications); and
- applicants that had already completed part of the qualification for which they applied for under the TSSA.
During the interview, the Interviewing Board asked all applicants to elaborate on the benefits that [they] would be acquiring upon successfully completing the Higher Qualification. Applicants had the opportunity to explain how the knowledge, skills and competences expected to be achieved through the Higher Qualification would affect their future research focus, employability or career prospects. Through these questions the Interviewing Board wanted to ensure that the applicants that are awarded a scholarship had the necessary aptitude to successfully complete the Programme.
The above methodology [was] in line with the objectives set under the Operational Programme II that encourage measures aimed at increasing the participation rate in lifelong learning and enable human resources to shift to those sectors where there is greater demand.”
In effect, complainant was awarded 28 marks out of a maximum of 50 for academic merit. The method used for assessing qualifications in this way was based on the distinction between ‘academic qualifications’ and ‘academic merit’, the latter concept being much wider than the paper qualifications themselves. This fact meant introducing a subjective element in the decision taken by the members of the Interviewing Board on the marks to be allotted to candidates for this criterion, depending on the subjective judgment of the members of the Interviewing Board on the replies/performance of the individual candidates during the interview.
In relation to complainant’s comment regarding ‘A’ Levels as against IT certificates, the Ombudsman noted that it did not result that possession of ‘A’ Levels was in any way determinant to the outcome of the result. There were in fact more successful candidates who were not in possession of ‘A’ levels than those who had ‘A’ Levels. Therefore he was not in a position to sustain the allegation that during the interviews the Board “…incorrectly gave the benefits to holders of A Levels even if irrelevant to the course material.”
It further resulted that the Interviewing Board also took into consideration complainant’s experience since June 2010 as IT Support Specialist as against the previous years as Technical Support, in relation to the course of studies applied for which in terms of the Scheme was intended for IT in gaming and financial services. Complainant was awarded 5.8 marks out of 10. Regarding performance during the interview, he obtained an average mark of 7.2 (out of a maximum of 10) which indicated a reasonably good overall performance at the interview in this respect.
In respect of motivation, the average marks awarded to complainant was 5.5 – a mark which was below average when the highest mark awarded for this criterion was 9.3. While the significance of the comment of the Appeals Board in respect of motivation (without underestimating complainant’s motivation to further his studies) was not understood in the context of the Interviewing Board having given him a below average mark, it had to be considered that this mark, as was the case of other criteria, depended on the subjective judgment of the Interviewing Board on how he replied to questions related to this issue during the interview.
Regarding the relevance of the choice of applicant’s study/the impact of work in the development of Malta, complainant had referred to his proposed course as aiming, amongst others, to provide an educational foundation for a range of technical software and that students who complete this diploma would find a number of career opportunities in ICT industry or in IT Departments. This was a valid argument and one would normally expect a reasonably high mark for all IT (Digital media) courses, especially when one considers the importance at national level given to training in IT. However, in evaluating complainant’s answers on this issue the Interviewing Board considered his replies as rather poor, resulting in a mark of 3.8 out of 10. Excluding one exceptional mark for one candidate, the highest mark that was awarded for this criterion was 8+. The comments in respect of this criterion apply also to the availability in the economy in respect of which complainant was awarded 4 marks. Save for one exceptional mark for the same candidate referred to earlier, the highest mark was 8.
In assessing complainant’s performance on these two criteria, the Board had the support of two technical experts. However, candidates had to present reasons/ arguments as to whether there was availability of such skills or not. The role of these experts was to assess whether what was being said by the candidates during the interview was correct or not. The Ombudsman was not in a position to know exactly what complainant said during his interview.
Overall complainant placed 219th when only the first 144 candidates in order of merit were awarded a scholarship.
Considerations
The Ombudsman considered that the Interviewing Board gave major importance to the way the candidates “proved their point” in explaining their case in respect of the selection criteria. While this was the reason for adopting the criteria of “academic merit” and not “academic qualifications”, the Board’s approach introduced a strong subjective element in the assessment of some criteria, in particular those of “Relevance of Study” and “Availability in Economy” where the course applied for by complainant would normally be expected to score higher marks than those awarded to him. In fact the vast majority of the candidates obtained a higher mark.
In this context it must however be considered that the Interviewing Board adopted this procedure across the board and there is no evidence of any differential approach in respect of the individual candidates. Once this was the case, it was not the function of this Office to substitute the subjective opinion of the members of the Board by his own, even if, for the sake of argument, he were to hold a different opinion. The same applies to any reviewing body, including, in this case the Appeals Board.
Without prejudice to the above, it had to be noted that even if theoretically each of complainant’s marks for the criteria “Relevance of Study” and “Availability in Economy” were topped up to a very good mark, for example 8 marks out of 10, this would have only added another 8.2 marks to his total and bring it to 62.27 marks. Complainant would only advance to 173rd place in order of merit when only 144 scholarships were awarded.
Conclusions
The Ombudsman therefore concluded that in light of the preceding considerations, namely:
i. that possession or otherwise of “A Levels” was not, in any way, determinant to the success or otherwise of applicants and in fact there were more successful applicants who did not have ‘A’ Levels than with such qualifications;
ii. the evaluation of candidates in respect of the pre-set criteria and related methodology was done subjectively and there was absolutely no evidence that this was not applied across the board and without distinction. It was not the function of the Ombudsman (or for that matter any reviewing body), to substitute the subjective judgment of the individual members of the Interviewing Board by his own, even if, for the sake of argument, he were to hold a different opinion; and even if, for the sake of argument only, one were to consider a theoretical upgrade (as indicated earlier in this report) of marks for two of the criteria where complainant was given a low mark, he would still not have qualified for a grant under the scheme.
The Ombudsman was therefore not in a position to sustain the complaint.
Case Studies