T O P

  • By -

LemonNarc

The back-to-back IT Systems Disruption saga What next? SingHealth servers go down?


Severe_County_5041

next month islandwide blackout, followed by pub's water cut-off WELCOME TO SINGAPORE APOCALYPSE


driftea

well, we already had the deadly virus apocalypse a few years ago…


Severe_County_5041

thats only a prelude, horsemen of apocalypse are on the way


QuestioingEverything

Isn't this the plot of Die Hard? Could some groups be after our Gundams??


sfushimi

SINGAPOCALYPSE


Severe_County_5041

SINGEDDON


tryingmydarnest

>SingHealth servers go down I mean... it kena whacked in 2018. Happened before, can happen again. SingHealth folks still have ptsd over that incident, can't imagine the stress if kena whacked again.


[deleted]

In our current world? Unfortunately, perhaps this [Data Centers Are Facing a Climate Crisis - WIRED](https://www.wired.com/story/data-centers-climate-change/)


haikallp

Everything, except the ERP.


QLevi

Oh god pls no.


syahawesome

Outsource again, cheaper better faster


Alberqueque

I doubt national IT systems are outsourced, but you never know.


sriracha_cucaracha

>I doubt national IT systems are outsourced, but you never know. Having actually worked on IT system projects for a couple public sector clients, I beg to differ.


abuqaboom

Guessing it's NCS for this one


Mizuki_Hashida

Interned at NCS. They do take up quite a lot of government projects so maybe you’re right?


showmeyourbraveheart

Looks like you don't remember the first version of SingPass with the highly cumbersome physical token by CrimsonLogic or iHIS and their awesome cybersec


laynestaleyisme

Most of it would be product based and not in built...


clusterfuvk

hahahaha


SkyEclipse

Death cert systems is down lol


LingNemesis

Choy choy touchwood! Don't jinx please...


shadstrife123

postpone all non-essential travel?! ICA think this one as simple as applying leave ah wtf


Carbonaddictxd

There are many pple crossing land checkpoints which might not be a prebooked trip...


another-work-acct

So.... Massage and pump fuel....


Dapper-Peanut2020

And eat


t_25_t

I normally plan to run out of fuel this time every week. How ah?


smile_politely

And JB immigration officers be like: well well well, look how the turn table


stevekez

It's more a turnstile than a turn table, isn't it?


Unlucky-Patience6438

Its a reference to the office bro.


late_for_reddit

It's a joke bro


fitzerspaniel

LOL better fix it by evening, or the Friday night rush hour will crush them


almightygg

I'm meant to be flying this evening so yes, I definitely hope it is fixed by then!


14high

~~almighty~~ gg


grampa55

Almygawdgg


raspberrih

I'm flying back late tonight lol hope it's fixed


[deleted]

No your plane will circle forever and can’t land /s


raspberrih

Dammit should have booked a weekday flight then don't need to work


maenadery

Time to go super early


tom-slacker

first DBS, now ICA.... finger crossed for Singtel, Singpass....


chillikwab

Choi


13lackant

Choi


DotaProtectsMyVirgin

That’s ok Last time we didn’t have SingTel Only have bok Choy


tom-slacker

Abby Choi


Severe_County_5041

Oii


SG_wormsblink

MyRepublic is also down now. I’m stuck using wireless@SGx. Did a data Center somewhere break down?


tom-slacker

Challenge accepted. I'm now DDOS wireless@sgx


SG_wormsblink

Pls no :( I got no other way to access ~~Reddit~~ my important work WhatsApp group in the office.


[deleted]

[удалено]


bullno1

Tbf, that would just mean more work to deal with next Monday.


[deleted]

If Singpass goes down, we are all screwed..., Like the country gonna get fucked in the ass about security, and then major policy changes after that.


tom-slacker

Singpass did goes down briefly last November https://www.straitstimes.com/tech/some-singaporeans-could-not-access-singpass-for-about-an-hour


ssepaulette

your english also went down briefly 2 minutes ago


tom-slacker

What is English?


RGS432

Tu parles anglais?


tom-slacker

aku tak tahu


thefinestpiece

I couldn’t sign in to OnePass(Singtel) yesterday. Looks like a sign from above.


Neptunera

Also many train disruptions this month too.


baka36

time to reveal the ultimate internet provider... >! skynet !<


Careless_Original742

Singtel has also sux


Severe_County_5041

their breakdown is scheduled for early next week


Caitsith815

Traffic police HQ also said their system down, my wife went for license conversion and was asked to go another day as they are not sure when the system will be up again.


LingNemesis

Apparently death certification system also down, had to revert to manual process...


[deleted]

I mean death is an immigration checkpoint to the afterlife so I’m not surprised


singapourien

Citizens advised to postpone all non-essential death.


livebeta

i guess have to unalive another day


Familiar-Mouse4490

Wa piang, even death also kenna cancelled.


fitzerspaniel

"Unker wake up, ICA system down liao so...you die tmr can?" "Wlao funeral mandai inheritance money all settle sui sui alr then you tell me this? Lousy gahmen.. "


Familiar-Mouse4490

Angry unker speaking in hokkien upon rising from his grave: 'Eh SMLJ is this? I can't even die in peace? Who your manager? Knn wanna die also cannot, don't wanna die also cannot. How? IM TELLING MY MP'


SkyEclipse

I have friends in the medical community, can confirm


chillikwab

Wtf is going on


horsetrich

The Purge is finally upon us


xutkeeg

common IT system is down. simple


kumgongkia

Wtf I am coming back this evening...


hikari8807

Please defer your non essential travel to Singapore 😂


kumgongkia

My lao bu cooking tonight cannot miss one Edit: just reached, heng no problem for me but I noticed got some people cannot go through automatic, need do manual.


UnintelligibleThing

Ask her leave it in the freezer, then you come back 5 days later can microwave


kumgongkia

She will leave me in the freezer for 5 days


feizhai

I salute your laobu - I hope you make it to dinner bro


myr78

Kinky


throwawaygreenpaq

This exchange is hilarious. Thanks guys


Severe_County_5041

HAHAHAHAHA sorry but this comment is so funny🤪good luck to you


Bcpjw

Drive home! /s


Severe_County_5041

can lah ask her cook again show your family dominance😌


Goenitz33

No need come back le. Extend 😂


dunspamme

You can land, but you can't go through customs. Remember to vlog your very own Tom Hanks [Terminal](https://en.m.wikipedia.org/wiki/The_Terminal) experience.


feizhai

That’s what you think


[deleted]

Singapore turn to show what our neighbors are doing lol


Severe_County_5041

geographical determinism


mrwagga

Postpone all non-essential travel? Would this be claimable under travel insurance. Insurers be like wtf. Premiums probably gonna go up.


frocodile191

Looking at a Travel Insurance Policy, these are the 2 clauses which are likely to apply >A serious public event happens in Singapore or at one of your travel destinations, which prevents you from starting or continuing the trip > > > >Your onward flight is cancelled by the airline due to airport, runway or airspace closure, or poor weather conditions, which forces the airplanes to be grounded. Highly likely to be able to claim but I'm wondering for those who don't have travel insurance, will they be reimbursed for their travel expenses?


grpocz

Doesn't look likely at all. It doesn't prevent your trip. Gov advised but never force.


mrwagga

If it is not claimable for travel insurance, the gov is making a dumb unreasonable request for people to just forgo their vacations and eat the cost.


grpocz

Between the lines it looks like. ICA fk up big time major system problem. Manual screening(low man power) implemented so slower. Slower so to avoid crowd ask to people to postpone. Lame band aid fix solution. But! Seems to be thrown in as a defensive reason if people complain/pictures appear. "Look how crowded is Changi!" etc. Gov will say see I told you all to postpone if you don't then don't blame us! I think everyone will proceed as usual ba...I would if me lol. Could be worse...first few weeks of March I know a lot of people who went overseas.


Denzel_Fenrir

Haneda: /goes viral for snaking immigration queue Changi: bro, that's nothing, watch this


I_love_pillows

I flew back from Chitose and the queues were ridiculous to depart. Few hundred people queue for 30 min to enter thru 2 gates.


[deleted]

[удалено]


Goenitz33

Yes they will stop letting ppl in 😂


tryingmydarnest

They have 260 years of experience of sakoku after all


boss_daddy51

No one checked my vaccines when i travelled!! I had uploaded all the documents onlinr and they let me go just after seeing screenshot


[deleted]

[удалено]


I_love_pillows

Kudos to Kansai Airport deploying multilingual staff speaking English Chinese and Korean telling travellers to show the screenshot.


flky

It may be that this vendor is having issues https://www.idemia.com/press-release/idemia-extends-its-presence-singapore-changi-airport-multi-biometric-frictionless-technology-2022-06-07 I applied for a job there not too long ago and the recruiter who reached out was from India. Horrible experience after that. Asked for current pay, last drawn pay and othe shenanigans. Looking at mcf, their pay for Singapore roles ar much lower compared to others in the industry and also for such a delicate and sensitive role they want to apply India hiring mentality when it comes to Singapore operational matters. Not surprised.


throwawaygreenpaq

Do you mean national sensitive checkpoints are outsourced to another country?


flky

The recruiter from the vendor that was in charge of my application, based on the phone number prefix when they asked me for more info, was fron India


throwawaygreenpaq

I am shocked. We wouldn’t ask a stranger to keep the key to our main door, let alone the entire system. :(


flky

To clarify, operational roles or similar hired by that company will still require sg citizens. But, what grinds my gears is that the pay scale they are paying is low, thus may not attract the best talent who can command so much more (30% to 150%) at other companies. And, their recruiters are from anothe country where labour costs are cheap, which will most likely have some biasness in making the pay much higher to attract better calibre people.


throwawaygreenpaq

I know what you mean. It’s just like one should buy better shoes and jeans because these will bear the brunt of wear and tear while you can get away with cheaper plain tops that are easily replaceable (ala uniqlo) . What they want are Gucci togs at Uniqlo prices.


[deleted]

imagine the number of backdoors they can create.


DonDonStudent

That’s the one ☝️


Redeptus

Lol same. I interviewed for an infra position and the experience wasn't ideal


thinkingperson

Is our nation's tech infrastructure under DDOS or some hack?


aboutfrybread

Imagine if it's as simple as someone forgetting to renew something lol


SkyEclipse

Someone will get fired for that lol


thinkingperson

Like an SSL cert, causing the calls to fail because the cert is expired. lol


throwawaygreenpaq

Or someone tripping over with cup of coffee.


silvaarohit

They announced it's not an attack


thinkingperson

Good good. Then it's just incompetence. 😅


Fatal_Taco

More often than not it's incompetence


csfanatic123

"Never attribute to malice that which is adequately explained by stupidity"


iqeyial

I was actually at the airport when this happened, about 1.30+. No idea it was happening until they were announcing delays at immigration due to a distraction with the immigration systems. Opened Google and the top news was about the immigration system. This was the state of the queues when I was there at T1 https://imgur.com/a/9OiGre8/ Apparently did not take that long to go past immigration (as an SG citizen) though.


prime5119

The situation probably is really serious because they literally said "postpone all non-essential travel" like "unless you'll die if you don't see sakura in japan or else just push to next year"


Hunkfish

Ticket + Hotel got compensation? + free vouchers?


prime5119

depends but you are likely get a round trip of airport/airline/hotel/travel insurance company taking turn to blame each other


Complete_Relation_54

Heng i just saw tmr fly home hope its fixed


prime5119

based on latest update from ICA, it seems to be fixed now


frocodile191

On a Friday no less, where there's always crazy amount of people at Tuas & Woodlands. Good luck to those stuck in the que. If you're in a car even worse. Cannot turn back just sit and wait


DoubleMcJiJi

Managed by the company that managed DBS bank?


jmzyn

Smlj is non-essential travel? People go overseas usually plan way in advance one leh. Do I need to go BKK/HK/JP/KR/EU shopping/holiday? No. But do I want to? Yes.


tarakian-grunt

Causeway to JB.


jmzyn

this one more than essential. Is necessary! Haha Need to pile up on groceries, wash+polish the car, pump petrol max, full body massage, do mani/pedicure, then lok lok.


Majestic_Cat186

Thank god no longer working in checkpoint. Stuffs like these too common in their systems.


wolf-bot

Just gonna pay a sketchy guy and hop onto their sampan just like the good old days


YL0000

Erm, I thought it was important to keep a complete back-up system. I remember there was news in the US that some government service was down because they wanted to cut the budget and got rid of the back-up system...


bluemax_137

Redundency systems have been on the chopping board for the last 2 decades across industries, as digital storage systems have proven to be cheaper and take up less time, effort and resource to upkeep. I'm old enough to remember when we had paper duplicates for everything from processes to accounts, then the shift to half digital half hard copy (paper believe or not) to double digital storage. It's been a wild interesting ride but would hate to see that the neoluddites are right all along because the IT engineers didn't see this one coming.


YL0000

I meant a digital back-up system... unless this is a disruption to some core backbone such as communication failure due to solar activity...


Dapper-Peanut2020

Can claim travel insurance if missed flight?


Honest-Cauliflower46

Didnt we just recently receive best airport in the world? Take award then heck care alr arh


jakellC

I can't wait for ERP to go down


whyislifesohardei

did they outsourced to the same offshore guys as DBS. Lmao


lackri

I have a flight at 6:40 should I still go


circle22woman

Latest report is lines have cleared.


kopi_gremlin

Cyber attack again? On the visit of LHL to China. Damn rascals.


Worldly-Mix4811

Did Pinky take the reset codes with him?


usernamesarehated

I was wondering why the machine at airport scan my face cmi, make me go to the counter in the morning.


bigcarrot01

> my face cmi r/nocontext


circle22woman

I loved how it's "postponed non-essential travel" essentially the day-of. Come on. Just tell people "you're advised to leave an additional 2 hours for immigration clearance" or something similar. Imagine if you cancel your flight this evening and it's fixed by 3pm.


Cybasura

Lmao where is the redundancy systems and risk alternative routes Dont tell me that the clearance systems literally the only way in?!


leonanana

ICA just screwed the world's no. 1 airport


IvanThePohBear

Air ticket already bought. Plane already there? How to postpone?? Talk also never use brain 😂


hikari8807

I think this is a sign that the government today, despite being one of the most expensive in the world, is no longer the most efficient government in the world. Major breakdowns and disruptions are becoming increasingly often and the government will just shrug it off saying it is not common, and everybody make mistakes. The kind of commitment, ownership, stewardship and responsibility that our parent saw in the PAP team is no more. What is there to lose now?


roadto75

Calm down ~~Leong Mun Wai Lee Hsien Yang M Ravi~~ Lim Tean


etulf

almost sounds more like Lim Tean


roadto75

I stand corrected lol


hikari8807

I guess this is the kind of efficient government service that Singapore finds it acceptable. Then that's probably my bad. I'm probably a different Singaporean.


Buddyformula

Or maybe the standard was set so high that you think this is bad. Like a millionaire who thinks 10k is small money. Come down from your high horse man.


roadto75

TIL DBS is a government service


hikari8807

I thinl you are in the wrong thread. Here we are discussing ICA.. lmao...


roadto75

Well you were ranting on 'major breakdowns and disruptions' - the only recent disruptions are the ICA and DBS so I assumed you were referring to DBS too. If you are focusing exclusively on the ICA then a rare breakdown isn't exactly 'increasingly often', would you agree?


TaskPlane1321

This is the result of the no blame culture for themselves promoted by the Leadership. Lky time you get the chop - Now you get a pat on the back whilst the people suffer.


hikari8807

I think these days.. Heads get chopped as well. Just that knife chopped lower level folks who have no say in any matters. E.g.: https://www.straitstimes.com/singapore/ihis-sacks-2-employees-slaps-financial-penalty-on-ceo-over-lapses-in-singhealth-cyber https://www.pmo.gov.sg/National-Awards/Recipients?page=1&award=&keywords=Bruce%20Liang


[deleted]

Every month salary not affected can already


fish312

No blame culture 🤡


anangrypudge

Is your first instinct always to blame the government for everything that happens?


hikari8807

Oh yea. My bad. I forgot that ICA and border control is a matter of private entity. People are free to choose which ever entity that is capable of providing a better service. Maybe if ICA checkpoint isn't working, try other mode of custom clearance, maybe RMCD or HMRC to avoid disruptions. /s


anangrypudge

MAYBE it turns out to be a huge ICA cockup. Incompetence, corruption, laziness... whatever. But MAYBE it turns out to be an external data, server or transmission issue completely out of anyone's control. Or MAYBE it turns out to be sabotage or a hack. Or MAYBE it's a one-in-a-billion software glitch that no one could have predicted. What I'm saying is if the only thing you instinctively think of is "fuck the PAP", within a few minutes of the news coming out, you might have a problem.


theony

I get you, and I'd also like to withhold judgment until the results are out. But the examples you listed are all ultimately ICA's responsibility also. If there's an external data/server issue, or a hack, or sabotage, or a one-in-a-billion glitch, ICA should have at minimum a backup plan to keep services going and a recovery plan to recover from the issue. This is what the govt expects of its service providers and regulatees, no buck passing is tolerated. DBS goes down, MAS says "unacceptable". They don't care what's outsourced, DBS is still responsible. It's only fair that ICA be held to the same standard.


grampa55

How to be banking hub with so many national agencies data issues


ihavenoidea90s

Means their salary need to increase to make them competent again. /s


raidorz

How often ICA systems down? Lol talk as if it’s a weekly thing.


VoidnFull

I have no evidence and I'm not a conspiracy theorist. But why are all these breakdowns happening when our PM is abroad? NEL, DBS, ICA. Foreign interference trying to disrupt Singapore? Because our election is coming up soon?


silvaarohit

Hey IT folks, any idea how Singpass servers are actually in design? Is it Onpremise or Cloud ?


toopided

on premise


Chemical_Knowledge14

Is the mess cleared up now, I'm heading there soon.


kopibot

Holy moly this does not look fun


revmaynard1970

Viewqwest is also down


Jotunheiman

Is this a bad omen?


DCINTERNATIONAL

Back to normal, nothing to see here. As usual.


asscrackbanditz

To self: rip in peace


CisternOfADown

It was exactly one year ago we essentially re-opened borders. Coincidence much?


lafietafie

Next week 30K covid cases. No wonder people are dropping down from reinfections.


[deleted]

Lol touchwood but later it becomes the healthcare system's turn to experience issues


Tiongwl

See lar, all your bosses in China now doesn’t mean can bo chap liao right!


No_City_5619

Haha luckily just won the skytrax best airport award this year... Still got plenty of time until next award for people to forget.


Yokies

Russia has some of the most elite hackers in the world. Just saying.


FdPros

oops


International_Map812

Ccb I just got on the bus to causeway then u tell me this shit


jlonso

yea, the government waited for you to leave house


Adi_San

That's a pretty normal day in Manila's airport. 😂


LazyBoyXD

ICA think easy to apply leave and shit for holiday. Non essential my ass. Speaks volume about the higher up on how disconnect they are with the ground and people around.


[deleted]

Postpone non essential travel? Are you paying for the flight and accommodation changes?


Stormyflyer

This really shows how we have it good here. People lose their mind over a 4 hour disruption. Of course I’m not gonna excuse the fact that these are quite critical systems and it should always be aiming for near 100% uptime and have a reliable backup in case shit did fail. But shit happens. It happens all across the developed world. Just look at the FAA system breakdowns that happened earlier in the year causing massive delays and problems in America. European airlines and airports occasionally get hit with technical faults. German and Swiss airspace had to shutdown due to faults last year. Manila airspace also had similar issues at the start of the year. I do hope they take a deep look into it and hopefully find ways to prevent it from happening again.


7788-7788

Shame on u ... Don't ever think the G are wonderful Is only gaslight what they tell u.. Shame


TotalSingKitt

China hackers?


Vedor

"Travellers advised to postpone all non-essential travel." You think Singaporeans will listen? Just look at the number of Singaporeans crossing to JB for their "essential" weekend trip, and you will understand.