Andrew is the CEO of the Real World Group, a family of IT and telecommunications companies based in Sydney, Australia. Andrew loves Jesus, his wife Bess and his four awesome kids.
The day where I celebrate a death. The death of a man. Moreover, I celebrate the death of God.
On a cross, 2000 years ago, the world was changed. When Jesus spoke those last words – “It is finished” – it changed history forever. Because it was finished. God himself died. Why?
Sacrifice.
His death was to pay the price for all of the evil in the world.
Only the death of the majestic unlimited ruler of the universe could be enough to sacrifice for the whole world.
For it is in this beautiful demonstration of servant hearted leadership and sacrifice that I am given hope.
It’s not hope of nothing. It’s not hope of an empty future. It’s a hope of forgiveness. It’s a hope of redemption. It’s a hope given because God himself stepped in to death for me. Willingly.
The story doesn’t stop here. His death was not the end.
At a meeting I recently attended, my friend Rachel astutely observed that we are often more likely to be open about what our thoughts, struggles and challenges are with those who have short term impacts on our lives. There is something to be said about the safety of being able to close off ourselves after a period of time.
But as I reflect on this, I’m confronted with the sadness that we feel compelled to shield our lives from those around us. We’re concerned that our vulnerability might cause people to think less of us. We’re concerned that others might judge us. We’ve learnt that being vulnerable gets us burnt, and so we choose to lock our feelings, thoughts and emotions away.
Days like today (it is RUOK day if you missed it) try to challenge this, by encouraging us to take the first step to really enquire as to the wellbeing of another. To genuinely care.
But as many of my friends have observed the challenge with this is that those who are genuinely struggling don’t just need those “moments” of connection; they need long term, ongoing support, encouragement and care. They need friendship from people who will not just suspend judgement of their feelings, but who will genuinely decide that the person who is experiencing the downs of life is more important than the feelings and emotion that surrounds it. They need people who will support them regardless of the situation and concern.
If you are going to ask RUOK today (and I would encourage you to do so), also take the time to assess your own reactions, thoughts and emotional responses to those you are trying to support. Try to recognise where your responses are judgemental rather than caring and compassionate. Challenge yourself to “check” those responses, and consider how you can build relationships where the person you are relating to is more important than the weight of experience they are going through. Consider how today can just be one step in a longer journey of care and support.
While it might be easier to share how you are feeling with someone who is only there for a moment, it would be so much better to share with someone who is going to be beside you for the long haul.
Supporting someone who is dealing with the dark side of life can be hard – but remember, your actions in supporting them are probably no where near what the person is going through.
Walk this road together. There is light in the darkness, even if we sometimes can’t see it.
As the parent of three children, many of you are aware that I am active within the school community. I am also an active supporter of the public education system, our local school teachers and the school environment.
I constantly praise the school for their work providing not just the academic education of my children, but also the emotional, social and physical education of my children. These aspects of their education also transcend the classroom into our homes, and then extend into our everyday lives, shaping and sculpting my children as they grow and develop.
It is in this spirit that I also support the teaching provided by SRE and SEE. The values taught through these curriculums shape and provide an appropriate lens for my children to understand the world in which they live in a manner that is congruent with the values we also hold at home.
I acknowledge that parents will hold a range of diverse views on how these are taught, and so believe it is appropriate for this education to be delivered through a framework that is compatible with the beliefs and teachings chosen by the parents. The approach of SRE and SEE within the NSW Education System provide this avenue.
However, I am concerned that if normal academic classes continue at the same time as SRE & SEE, my children will be disadvantaged as they will not be able to participate in aspects of their schooling. I am also concerned that parents, including myself, will feel compelled to make a choice between two vital areas of education – even if they strongly believe they should not be in opposition to one another.
Having said this, I acknowledge that some parents within my school community may hold a different view to me. I respect their right to a diversity of opinion regarding the value of SRE and SEE, and support their right to hold these opinions and continue the public discourse regarding this.
I am concerned that the P&C Federation view, as contained within the media release, cannot represent the diversity of opinion that I know exists within our school community.
As such, I am writing to you to ask:
That the P&C Association acknowledges that parents may hold a diversity of opinions regarding SRE & SEE, their place within the school system, and how they should be delivered
That the P&C Association acknowledges that it is not appropriate for it to form a view on behalf of the parent community unless it can demonstrate that the view is held by the parent community as a whole
That, in this vein, the P&C Association writes to the NSW Federation to indicate that:
Our parent community holds a wide variety of views regarding SRE & SEE
The association feels that it is not appropriate for the Federation to make a public statement regarding the view of their membership and affiliated bodies when a significant number of them are actually not in support of the position statement
the Federation to publicly acknowledge that they are not in a position to make a statement regarding this matter due to a range of diverse opinions regarding SRE and SEE throughout their schools and members
the Federation refrain from making public policy comment regarding SRE & SEE unless it can demonstrate that the member parent body is broadly in support of its position
I am writing to you to express my support and sincerest thanks for the resumption of SRE & SEE within our school.
While I acknowledge that you are acting in line with departmental guidelines, I appreciate that a decision to resume any school activity in the current climate is difficult.
My children have been very excited by the return to SRE and have spoken very positively of their experience since the resumption of the program.
It is no secret that I believe SRE & SEE are vital compliments to the education that is delivered within our school. It is also no secret that I am strongly in support of the continued presence of the delivery of strong values based teaching that SRE & SEE provides, especially as it complements the regular teaching program.
I remain genuinely thankful for your continued support of the SRE & SEE programs within our school.
This week has reminded me that many of us are not OK.
Even if we ourselves aren’t affected, many of us are grieving for those around us; what we have lost, and what we expect to lose. We’re hurting for those around us who have lost their jobs, their livelihoods, and their futures. We’re tired, and worn, and run down.
We’re grieving for the plans we made that now are forfeit. We’re grieving the moments lost. The small snippets of our days that we can no longer enjoy. The basic freedoms we took for granted that we never thought we’d lose.
Some of us are afraid for us, our families and our friends; and recognise both the rationality and irrationality of it. We’re angry, we’re sad, we’re hurting and we’re worn.
We’re thankful for where we are, but not sure how we will find the energy for where we have to go. Whether that be go tomorrow, next week, or next month or longer.
I’ve found myself saying the words “it’s just for a time” more often than not over the last few days. I’ve found myself pointing that there will be an end; and at the same time doubting that it will come; or that it will come soon enough. The grief of this uncertainty, even in my optimism, is hard.
In my conversation with another parent from school tonight, reflecting on the impact of COVID, we mused together that our shared hope is not for the things of this life; and that this year, or next, is just a drop in eternity. And then my sister reminds me tonight, once again, that even in the pain, even in the hurt, even when things are looking sad and hard, are blessings as we learn to persevere, and trust.
And yet, even despite this hope, and how much it moves me to tears tonight; I still grieve over what we have lost and what we are losing. I grieve over the impact that this is having day to day on my friends and my family. I grieve over the uncertainty of the future.
And for now, that is ok. And it’s ok for you to grieve too. It’s ok for you to feel like the world is an uncertain mess. It’s not wrong to feel these things. And it’s OK to share our fear.
And yes, some people may stamp on your concerns. They may belittle your fears, even accidentally. They may shut you down, and leave you with nothing but a sense of guilt for even attempting to be open.
But there are others of us that are grieving with you; and we will share your tears, even as we shed our own.
Christian friends in production or ministry, I’m considering putting together a webcast to cover some of the do’s & don’ts of streaming for churches; picking up some of the practical and pastoral tips I (and others I’ve worked with) have learnt.
Would you be interested in contributing or watching once done?
ReFS is Microsoft’s new file system that will one day replace NTFS. It offers many awesome new features, particularly if you are using Storage Spaces and lots of disks. It scales beautifully, and has no fixed capacity limitations that matter in this day and age.
I’m no technical expert on ReFS, but we’ve recently run into an issue with an ReFS Cluster Shared Volume in Windows 2019 that was nice and yucky. Essentially, we’d been experiencing some issues with a Windows 2019 Hyper V Cluster resulting in storage becoming unavailable. The error generated was a useless generic “An Unexpected Error Occurred.” We opened a support case with Microsoft, and after some basic testing came to the conclusion that the error might have been caused be a specific registry setting, and so brought our storage online.
Except that it didn’t come online. Instead we got hit with a classic ReFS RAW Volume; essentially a disk that Windows could see, but was unable to mount. Doing some investigation, it looked like all the data was there, but just the metadata was obstructed in some way. Windows provides “really helpful” error messages:
Attempting to open the volume in Explorer provides this immensely helpful error message.
Entirely helpful messages from the Event Log
We really wanted to get the data that was on this disk to avoid having to roll back to an earlier backup.
After much anxiety (and googling) we decided to try a few strategies to get the data back. First of all, we tried ReclaiMe. ReclaiMe is a commercial tool that allows you to do data recovery from a number of volumes. ReclaiMe started out really well. It found all the files we were expecting to find on the volume, and displayed them in the tree. We provisioned a new NTFS volume and ran the data recovery; however the majority of the data we recovered was unreadable. We could see that it should be working, but couldn’t understand why it wasn’t.
I did a lot more reading on ReFS. It turns out that MS has several versions of ReFS, and the current version is 3.4 (as of Windows 2019 and the Windows 2016 1803 update). ReclaimMe identified the disk as a ReFS 2 volume. While the differences don’t necessarily explain why it might not work, I had a hunch that maybe the reason the restore wasn’t working correctly was because we were in fact on too new a version of ReFS for the ReclaiMe software to work.
A few weeks ago, Anton Gostev from Veeam wrote about a new, otherwise undocumented tool in Windows 2019 called “refsutil”. This tool provides a mechanism to triage and recover failed ReFS volumes. The post in the Veeam digest indicated that it existed, was probably good, but no one really knows what it does. There is almost no documentation or information about this tool, but it saved the day for us – and so I thought it was worth noting down some useful things we learnt about how to use the tool. Much of our learning was greatly assisted by this article.
You can see all of the options available in ReFSutil for data salvage by running “refsutil salvage” with no options.
Knowing that our volume had data, the first thing we wanted to do was to verify if ReFSutil could see the corruption. Fascinatingly, ReFSutil thought everything was fine:
C:\salvage>refsutil salvage -D E: C:\salvage -x -v
Microsoft ReFS Salvage [Version 10.0.11070]
Copyright (c) 2015 Microsoft Corp.
Local time: 4/21/2019 2:29:57
Option(s) specified: -v -x
ReFS version: 3.4
Boot sector checked.
Superblocks checked.
Checkpoints checked.
No corruption is detected.
Command Completed.
Run time = 7 seconds.
To explain, “-D” says “diagnose” why the volume is failed. “E:” is the drive that was not mounting (our corrupted ReFS volume). “C:\salvage” was the directory where we were storing metadata about the recovery process. “-x” means “unmount the volume” before we go (if we didn’t do this we had access denied errors) and “-v” means be verbose about the output.
As described in the output above, there were no issues on the volume. Yet Windows would not mount it. This gave us a high degree of confidence we probably could recover the data. It also gives us a high degree of confusion as to what is actually wrong with the volume.
The next step was to run a Quick Scan to pull out all of the metadata we needed, and a list of files that ReFSutil was comfortable we could recover.
C:\salvage>refsutil salvage -QS E: C:\salvage\ -v -x
Microsoft ReFS Salvage [Version 10.0.11070]
Copyright (c) 2015 Microsoft Corp.
Local time: 4/21/2019 2:27:17
Option(s) specified: -v -x
ReFS version: 3.4
Boot sector checked.
Cluster Size: 65536 (0x10000).
Cluster Count: 486601728 (0x1d00f400).
Superblocks checked.
Checkpoints checked.
4363 container table entry pages processed (0 invalid page(s)).
1 container index table entry pages processed (0 invalid page(s)).
Container Table checked.
Processing 1 of 2 object table pages (50%)...
Object Table checked.
Examining identified metadata disk data for versioning and consistency.
9104 disk clusters analyzed (200%)...
Examining volume with signature a0e4914d for salvageable files.
8726 container table entry pages processed (0 invalid page(s)).
2 container index table entry pages processed (0 invalid page(s)).
Validating discovered table roots on volume with signature a0e4914d.
86 table roots validated (100%).
Enumerating files from discovered tables on volume with signature a0e4914d.
86 tables enumerated (100%).
Command Completed.
Run time = 22 seconds.
With regards to the options, “-QS” said perform a “quick scan” to look for files on the disk. There is also a Deep Scan option that will scan on a block-by-block basis for data. We didn’t believe we needed this as there was actually no evidence of actual corruption. As before, “E:” is the volume that was not mounting, “C:\salvage” is the location we were saving our working data, “-x” meant unmount the volume before we begin and “-v” meant be verbose about the output.
This ran successfully, and created a bunch of output in our working directory.
Of these files, the most useful for us is the list of files that ReFSUtil thought it could recover. Here’s a sample of what that looks like:
One of the important things to note is that you can edit this file to build a subset list of files you want to restore; which means you don’t need to do everything at once, or can prioritise key systems and data.
Here is an example of just restoring the two files in the index above:
C:\salvage>refsutil salvage -SL E: C:\salvage F:\restore C:\salvage\restorefiles3.txt -v
Microsoft ReFS Salvage [Version 10.0.11070]
Copyright (c) 2015 Microsoft Corp.
Local time: 4/21/2019 2:56:57
Option(s) specified: -v
Processing C:\salvage\restorefiles3.txt
8726 container table entry pages processed (0 invalid page(s)).
2 container index table entry pages processed (0 invalid page(s)).
Copying: \\?\F:\restore\volume_a0e4914d\VMSMB01\VMSMB01_TimeMachine.vhdx...Done
Copying: \\?\F:\restore\volume_a0e4914d\VMSWAN01\Virtual Hard Disks\VMSVWAN01_D.vhdx...Done
Command Completed.
Run time = 30125 seconds.
“-SL” means copy all the files in the “Source List”. “E:” is again our corrupted volume. “C:\salvage” contains the metadata we extracted in the “QS” step. “F:\restore” is where we are putting our recovered data copies. “C:\salvage\restorefiles3.txt” is our edited data set containing the files we wish to restore, “-x” meant unmount the volume before we begin and “-v” meant be verbose about the output.
From here, we were able to reattach the disks to the virtual machines, and win!
So what did we learn?
It’s probably not worth paying for a commercial data recovery tool for most ReFS failure scenarios. The built in ReFSutil is powerful, current and works for even recent versions of the ReFS file system
Even if Microsoft Support tells you everything is OK, it’s probably worth double checking that it actually is. Sometimes they are wrong.
Make sure you have current/recent backups. (We did, but we wanted to get the most recent data.)
Think twice before you use ReFS in a cluster file system; the tools and techniques for dealing with problems relating to it are not as robust as those for other file systems – and we still have no root cause/reason for the corruption that occurred.
Make sure you have tested the escalation process on your MS support case before your engineer goes off-shift, in case it’s magically broken.
(Many thanks to Lachlan, Dave and Dave who were instrumental in the process of puzzling this out.)